A major bottleneck in millimeter wave hybrid beamforming (BF) system is the limited number of radio frequency (RF) chains at the base station (BS). For example, when the number of users is large, the joint user scheduling and BF design is needed under the hardware constraint. Fortunately, if multiple RF chains are available at both BS and users, the system performance can significantly be improved by multi-mode transmission, which assigns different numbers of streams to the users. However, the optimal multi-mode scheme may require high feedback-feedforward overhead and complexity. Furthermore, since the mmWave BF system operates in a large bandwidth, the wideband channel should be considered. To tackle this problem, we propose the single-carrier (SC) hybrid BF scheme based on the limited feedback information. We maximize the weighted sum rate by adopting the multi-mode transmission based on the RF beam alignment. The proposed SC hybrid BF scheme outperforms the existing orthogonal frequency division multiplexing (OFDM)-based hybrid BF schemes. From the analysis, we find that full spatial multiplexing gain is guaranteed if we scale the number of RF chains at the BS as smaller than the square root of the number of antennas in a large number of users regime.