In this paper, a novel distortion model based on a mixture of Laplacian distributions is presented for the transform coefficients of predicted residues in quadtree coding. The mixture Laplacian distribution is made on the coding structure with different quadtree coding unit (CU) depth. Moreover, for intra-coded CU, the distortion model is asymptotically simplified based on the signal characteristics of the transform coefficient. The proposed mixture model of multiple Laplacian distributions is tested for the High Efficiency Video Coding (HEVC) Test Model (HM) with quadtreestructured Coding Unit (CU) and Transform Unit (TU). The experimental results show that the proposed model achieves more accurate results of distortion estimation than the single probability models.