Accurate and precise navigation solution can be obtained by integrating multiple sensors such as global navigation satellite system (GNSS), vision sensor, and inertial navigation system (INS). However, accuracy of position solutions under GNSS-challenged environment occasionally degrades due to poor distributions of GNSS satellites and feature points from vision sensors. This paper proposes a selective integration method, which improves positioning accuracy under GNSS-challenged environments when applied to the multiple navigation sensors such as GNSS, a vision sensor, and INS. A performance index is introduced to recognize poor environments where navigation errors increase when measurements are added. The weighted least squares method was applied to derive the performance index, which measures the goodness of geometrical distributions of the satellites and feature points. It was also used to predict the position errors and the effects of the integration, and as a criterion to select the navigation sensors to be integrated. The feasibility of the proposed method was verified through a simulation and an experimental test. The performance index was examined by checking its correlation with the positional error covariance, and the performance of the selective navigation was verified by comparing its solution with the reference position. The results show that the selective integration of multiple sensors improves the positioning accuracy compared with nonselective integration when applied under GNSS-challenged environments. It is especially effective when satellites and feature points are posed in certain directions and have poor geometry.