In this paper, a novel loss function of vanishing point measurements for line-based simultaneous localization and mapping (SLAM) is proposed. In general, the Huber norm is used as loss functions for point and line features in feature-based SLAM. Because the point and line feature measurements define the reprojection error in the image plane as a residual, the loss function such as the Huber norm is used. However, the typical loss functions are not suitable for vanishing point measurements with unbounded problems. To tackle this problem, we propose a loss function for vanishing point measurements. The proposed loss function is based on the unit sphere model. Finally, we prove the validity of the loss function for vanishing point through experiments on a public dataset.