Learning from Imbalanced and Hierarchically Dual-Labeled Data

Abstract

The success of deep neural networks depends strongly on the availability of balanced and well-refined datasets, but in a real industry or practical cases, it is difficult to create such a dataset. In this paper, we present an approach to effectively use a highly imbalanced, hierarchically dual-labeled and multi-channel dataset: 1) using difference images, 2) transfer learning, 3) data argumentation and 4) DNN model based on an autoencoder which has the same number of bottleneck nodes as the number of classes. We evaluate our approach on the HDI AFVI dataset, containing 65k 4 channel images, 37 classes, and dual-labeled for each 36 defective class. The result demonstrates the effectiveness of our proposed methods in realistic settings.