Self-supervised representation learning for visual anomaly detection

Abstract

Self-supervised learning, a form of unsupervised learning, allows for the better utilization of unlabelled data by setting the learning objective to learn from the internal cues. In self-supervised learning, a pretext can be formulated by using the unlabeled data. The network while solving these pretext tasks learns a useful feature representation. This learned feature representation carries rich semantic and structural meaning which can be used in downstream tasks such as classification, object detection, segmentation, and anomaly detection. While classification, object detection, and segmentation have been investigated with self-supervised learning, anomaly detection needs more attention. We consider the problem of anomaly detection in images and videos, and present a new visual anomaly detection technique for videos. Numerous seminal and state-of-the-art self-supervised methods are evaluated for anomaly detection on a variety of image datasets which include CIFAR-10, CIFAR-100, fashion-MNIST, and ImageNet. Overall best performing image-based self-supervision method is also used in video anomaly detection to see the importance of spatial features. We then propose a new self-supervision approach for learning temporal coherence across video frames without the use of any optical flow information. At its core, our method identifies the frame indices of a jumbled video sequence allowing it to learn the spatiotemporal features of the video. This intuitive approach shows superior performance of visual anomaly detection compared to numerous methods for images and videos on UCF101 and ILSVRC2015 video datasets.