Delving into semi-supervised learning with ranking loss of metric learning

Abstract

Semi-supervised learning (SSL) is a research field playing a vital role in leveraging unlabeled data. In many applications especially real-world problems, labeled data is often very difficult and costly to obtain, so semi-supervised learning is significantly meaningful in fulfilling the limitation of labeled data. One of the most successful SSL approaches is consistency regularization, encouraging the model to produce unchanged when the input is perturbed. We argue that the inputs from the same class should also have the similar model outputs. Therefore, we unify the idea of consistency regularization SSL approach and metric learning to propose a more powerful SSL method, RankingMatch, for image classification. Our method considers both the perturbed inputs and the similarity among the same-class inputs, encouraging the model to produce the similar outputs for not only the different perturbations of the same input but also the inputs from the same class. Delving into the objective function of metric learning, we introduce a novel objective function, BatchMean Triplet loss, which has the advantage of computational efficiency while taking into account all inputs when computing the loss. Our method achieves state-of-the-art results across many standard SSL benchmarks with various labeled data amounts, including 95.13% accuracy on CIFAR-10 with 250 labels, 77.65% accuracy on CIFAR-100 with 10000 labels, 97.76% accuracy on SVHN with 250 labels, and 97.77% accuracy on SVHN with 1000 labels. We also conduct an ablation study, visualizations, and computational measurements to prove the efficacy of our proposed BatchMean Triplet loss in both terms of classification and computational efficiency.