We theoretically investigate the membrane fluctuations of red blood cells with focus laid on the role of the cytoskeleton, viewing the system as a membrane coupled to a sparse spring network. This model is exactly solvable and enables us to examine the coupling strength dependence of the membrane undulation. We find that the coupling modifies the fluctuation spectrum at wavelengths longer than the mesh size of the network, while leaving the fluid-like behavior of the membrane intact at shorter wavelengths. The fluctuation spectra can be markedly different, depending on not only the relative amplitude of the bilayer bending energy with respect to the cytoskeleton deformation energy but also the bilayer-cytoskelton coupling strength.