The present invention relates to a process for preparing a composite inorganic membrane for hydrogen separation, more specifically, to a process for preparing a composite inorganic membrane for hydrogen separation which comprises formation of intermediate layer in an asymmetric porous alumina support by sol-gel method, and modification of pores of the alumina membrane by soaking palladium acetate solution into the pores and vapor-deposition to enhance the permeability and selectivity for hydrogen. A process for preparing an inorganic membrane for hydrogen separation comprises the steps of: (i) dip-coating an alumina support in alumina sol containing a palladium precursor which is uniformly dispersed while vacuumizing the inside of the support, to form an intermediate layer of thin palladium impregnated alumina membrane; (ii) soaking the membrane obtained in step(i) in the palladium precursor solution, drying it under vacuum for 2 to 4 hours, and vaporizing the soaked materials with the gradual increase of temperature up to 170.degree. to 190.degree. C. for the vaporized palladium precursor to be deposited onto the pores; and, (iii) heating the alumina membrane whose pores are deposited with palladium at 290.degree. to 310.degree. C. while vacuumizing the inside of the support.