Conventional most probable point (MPP)-based dimension reduction methods (DRM) show better accuracy in reliability analysis than first-order reliability methods (FORM), and thus have been successfully applied to reliability-based design optimization (RBDO). However, the MPP-based DRM requires additional function evaluations to improve accuracy of probability of failure estimation which could be computationally expensive, and thus leads to reduction in efficiency. Therefore, in this paper, we propose an MPP-based approximated DRM (ADRM) that performs one more approximation at MPP to pursue accuracy of DRM while maintaining efficiency of FORM. In the proposed method, performance functions will be approximated in the original X-space with simplified bivariate DRM and linear regression using available function information such as gradients obtained during previous MPP searches. Therefore, evaluation of quadrature points can be replaced by the proposed approximation. In this manner, we eliminate function evaluations at quadrature points for reliability analysis, so that the proposed method requires function evaluations for MPP search only, which is identical with FORM. In RBDO where sequential reliability analyses in different design points are necessary, ADRM becomes more powerful due to accumulated function information, which will lead to more accurate approximation. To further improve efficiency of the proposed method, several techniques, such as local window and adaptive initial point, are proposed as well. Numerical study verifies that the proposed method is as accurate as DRM and as efficient as FORM by utilizing available function information obtained during MPP searches.