Error Backpropagation with Attention Control to Learn Imbalanced Data for Regression

Abstract

The imbalanced data problem refers to a skewed distribution of data over each class. Learning with imbalanced data constitutes a significant challenge for both research and industry applications. This often impairs performance of even powerful machine learning algorithms, such as Deep Artificial Neural Networks (DNN). To alleviate this problem, this paper proposes a neuroscience-inspired approach to train DNN, in which each neuron learns to filter the input while minimizing the training error though error back propagation. The proposed framework possesses two important structural and functional characteristics of biological vision systems: (1) each layer of the visual hierarchy has a non-uniform receptive field distribution and (2) the visual cortex learns to control its attention during development. The learned filter makes each neuron react only to a specific input range, encouraging some neuron committed to learn imbalanced samples. We demonstrate that the proposed model achieves improved performance in various imbalanced data-learning scenarios.