Dimension reduction techniques for deep novelty detection

Abstract

In this thesis, techniques on deep novelty detection on various data domains are discussed. The thesis consists of two parts.

For the first part of the thesis, deep novelty detection on image datasets is discussed. Specifically, we deal with the situation where the dataset is unlabeled. Compared to the previous works, the contribution of our work is summarized as follows. First, we analyze the pathological phenomenon where the conventional deep novelty detection models often assign lower uncertainty on the out-of-distribution data. For the analysis, we propose a novel metric, an effective rank, that measures the complexity of the data. Second, based on our analysis, we propose a novel out-of-distribution detection model, SVD-RND, that explicitly discriminates over blurred images. Experiment results show that SVD-RND greatly improves over conventional novelty detection methods. Finally, we show that SVD-RND can be applied in various scenarios that include when there is no OOD validation data.

For the second part of the thesis, deep novelty detection on general datasets is discussed. Specifically, we research the computationally efficient data augmentation method for self-supervised learning in the general data domain. First, we show that conventional self-supervised learning methods require an excessive number of augmentations to perform. Furthermore, we propose a novel data augmentation method, PCA-PER, that employs principal component analysis and permutation for efficient data augmentation. Experiment results show that PCA-PER can perform robustly with the number of augmentations from 4 to 8 times less than the conventional data augmentation methods. In addition, when the dimension of data is very small, PCA-PER can be merged with conventional data augmentation techniques to show better results.