We investigated the loss modulation effect on the second-harmonically mode-locked erbium-doped fiber laser. based on the Sagnac loop reflector with a Y-branch LiNbO3 phase modulator. We found that even a small loss modulation originating from the phase modulator could cause severe peak power difference between two output pulses in a modulation period. It was also found that the peak power difference was sensitively dependent on the modulation frequency. By numerical simulation, we quantitatively analyzed the relation between the peak power difference and the amount of loss modulation, which agreed well with the experimental results.