On-the-fly data augmentation for facial expression classification by using D-GAN with limited data

Abstract

Collecting large-scale dataset to train deep neural networks is burdensome. Since the advent of Generative Adversarial Networks (GAN), GAN-based models that could generate photo-realistic images have been proposed. Because these generated images can be used as an augmented data to aid in training a classifier, GAN-based models could mitigate the necessity of collecting large-scale dataset. Recently, differential generative adversarial networks (D-GAN) has made GAN-based data augmentation more useful because D-GAN could be trained with even a small amount of training dataset. Conventional GAN-based data augmentation methods followed a two-step framework that generated images through a trained generator and trained a classifier with the generated images. In this process, various engineering questions (how many should we train a generative model and a classifier? or how many should we generate images?) need to be solved and memory space for storing the generated images is needed. In this paper, we propose an on-the-fly data augmentation that simultaneously train the generator and classifier. The on-the-fly data augmentation induces the effect of data augmentation in order to train the classifier during training the generator.