Pseudo-Labeling Using Gaussian Process for Semi-supervised Deep Learning

Abstract

The goal of semi-supervised learning is to improve the performance of supervised learning tasks using unlabeled data. Deep learning has a high demand for making use of large-scale unlabeled data. We propose a simple and novel method of utilizing unlabeled data for semi-supervised learning to improve the performance of the deep learning model. First, we train a Gaussian process classifier (GPC), and use its output on unlabeled data as a pseudo-label. Then, we pre-train the deep learning model with the pseudo-labeled data to initialize the parameters of the model. Finally, we fine-tune it with the labeled data. We apply this method to five classes of video from the UCF101 data set which contains a small amount of labeled data. The experiment results show significant performance improvements compared to those of the deep learning model without pre-training. We further study the pre-training effect using pseudo-labeled data with a different probability, and give some advice for practical application. Moreover, we propose to train GPC with stratified sampled data (SGPC) to reduce the computation time of GPC when there are large amounts of data. Finally, we generalize proposed pseudo-labeling method to any classifier having good performance and give some advice for pseudo-labeling method selection.