BaMgAl10O17:Mn phosphor particles with a spherical shape and fine size of less than 1 mu m were prepared by spray pyrolysis, and their luminescence properties under vacuum ultraviolet (VUV) excitation were investigated by changing the concentration of activators and the composition of host matrix. It was found that by substituting Sr for Ba sites in the conduction layers the excitation efficiency at wavelengths shorter than 160 nm is enhanced. As a result, the photoluminescence intensity of BaMgAl10O17:Mn phosphor increases under VUV excitation (147 nm) without any shift in the peak position of the emission spectrum. The photoluminescence intensity of (Ba,Sr)MgAl10O17:Mn phosphor was also additionally improved by codoping Eu2+ ions as a consequence of the energy transfer from Eu2+ in the conduction layer to Mn2+ in the spinel block. Finally, the highest photoluminescence intensity achieved when 40 at.% of Ba was replaced with Sr and 0.1 at.% Eu was codoped was about 20% higher than that of commercial BaMgAl10O17:Mn phosphor prepared by a conventional solid-state route.