For the optical testing of a large mirror with a long radius of curvature, it is generally necessary to use a single-shot phase-shifting interferometer to take several measurements because the influence of air turbulence on air stratification prevention should be reduced for an accurate measurement. In this paper, a new technique that applies hierarchical clustering, which classifies a few clusters according to the pattern similarity of acquired wrapped phases, is proposed. As the minority patterns of the wrapped phases are excluded, the effects of unknown noises can be reduced. For each cluster, the circular mean is used to calculate the denoised wrapped phases. The surface figure is obtained from the unwrapped phases. Because the running number of the phase-unwrapping algorithm becomes the same as the number of chosen clusters, the proposed technique is much faster than conventional procedures. In 500 measurements using a spherical mirror with a diameter of 1.5 m, the measurement time and standard deviation of the surface figure's root-mean-square error are reduced by up to approximately 23% and 15%, respectively, of the results obtained via the conventional procedure.