Mitigating spurious correlation for deep learning: weakly supervised and semi-supervised approach

Abstract

Neural networks often learn to make predictions that overly rely on spurious correlations existing in the dataset, which causes the model to be biased. Previous work tackles this issue by using explicit labeling on the spuriously correlated attributes or presuming a particular bias type. To bypass such costly supervision on the spurious attribute, we focus on developing a weaker form of supervision. We propose a weaker form of supervision, weakly supervised, and semi-supervised approaches to mitigate spurious correlation for classification and image generation. For classification, we first utilize a cheaper yet generic form of human knowledge, which can be widely applicable to various types of bias: reliance of neural networks on spurious correlation is most prominent during the early phase of training. We then propose a semi-supervised approach based on spurious attribute estimation to bridge the performance gap between weakly supervised and fully-supervised approaches. Finally, for image generation, we leverage a biased classifier to characterize the spurious correlation in the generative model as a weaker form of supervision. We then encourage the generator to synthesize minority samples that conflict with the spurious correlation.