In applying the spectral stochastic finite element methods to the frequency response analysis, the conventional methods are known to give unstable and inaccurate results near the natural frequencies. To address this issue, a new sensitivity based stabilized formulation for stochastic frequency response analysis is proposed in this paper. The main difference over the conventional spectral methods is that the polynomials of random variables are applied to both numerator and denominator in approximating the harmonic response solution. In order to reflect the resonance behavior of the structure, the denominator polynomials is constructed by utilizing the natural frequency sensitivity and the random mode superposition. The numerator is approximated by applying a polynomial chaos expansion, and its coefficients are obtained through the Galerkin or the spectral projection method. Through various numerical studies, it is seen that the proposed method improves accuracy, especially in the vicinities of structural natural frequencies compared to conventional spectral methods.