Guiding deep neural networks via self-supervised learning for discriminative and generative modeling

Abstract

Self-supervised learning, which learns by constructing artificial labels (a.k.a, self-supervision) from only the input signals, has recently gained considerable attention for learning semantic representations from unlabeled data. This dissertation investigates how to utilize such self-supervision for improving deep neural networks under various scenarios: supervised learning, transfer learning, and generative modeling. Specifically, we first propose a novel label augmentation technique using self-supervision for various fully-supervised settings, including few-shot and imbalanced classification. Second, we construct effective augmentation-aware self-supervision for improving the transferability of representations under various transfer learning scenarios. Last, we introduce a novel training framework of latent-variable energy-based models guided by self-supervised learning, which significantly improves the generation quality. Namely, we show self-supervised learning can improve both discriminative and generative modeling in deep learning.