Multi-layer graph matching methods effectively solve multi-attributed graph matching problems based on the multi-layer structure adopted to address the ambiguity and uncertainty arisen from the attribute integration. However, despite of its effectiveness for matching multi-attributed graphs, the approach has a long way to apply in the practical environment due to the scalability problem caused by a huge matrix for describing the multi-layer structure. In this paper, we propose a novel multi-layer graph matching algorithm based on the multi-layer graph factorization to address the issue. The main contribution of this research is three-fold. First, we propose a factorization method that decomposes the huge multi-layer matrix into several small matrices for efficiency. Second, we reformulate the original multi-layer matching problem into two relaxed problems by using the factorized matrices. Third, based on the relaxed problems, we propose a multi-layer graph matching algorithm inspired from the convex-concave relaxation procedure. In our extensive experiments on the synthetic and real-image datasets, the proposed method exhibits better performance than state-of-the-art algorithms based on the single-layer structure.