In this paper, we propose a two-stage vector channel quantizer for multiple-input multiple-output (MIMO) broadcast channels with limited feedback. When the number of total users is larger than the number of transmit antennas, the users to be served are selected and then the selected users are supported by beam-forming based on quantized feedback information. If channel gain information (CGI) and channel direction information (CDI) are independently quantized with a product codebook, as in conventional channel quantization, we show that the effect of CDI quantization errors is boosted by the CGI value, which becomes more pronounced as the number of users increases because the selected users are likely to have larger CGI than the others. Motivated by the analysis, we devise a new two-stage quantizer where CGI is quantized at the first stage, and then CDI is adaptively quantized at the second stage according to the quantized CGI value and total number of users. We optimize the two-stage quantizer for an arbitrarily given CGI quantizer and the number of total users. It is demonstrated that for the same feedback size, the proposed quantizer markedly improves conventional quantization with a product codebook in terms of average sum rate in MIMO BC.