Domain adaptation and semi-supervised learning approaches to the data scarcity problem in computer vision

Abstract

Since deep learning-based supervised learning methods have been in the spotlight, the time and monetary costs of the ground-truth generation process and the data scarcity problem in computer vision have been a problem due to the excessive dependence of supervised learning methods on the annotated data. Due to the degenerative performance issue caused by domain discrepancy between training and test environments, the data scarcity problem further hinders the generalization of models to a new environment. Generalization approaches that tackle such data scarcity problem in computer vision can be classified into three types: i) Dependency reduction approaches on ground-truth, ii) Transfer learning approaches, and iii) learning approaches with noisy data. In this dissertation, we focus on the semi-supervised learning approaches of the ground-truth dependence reduction methods and the domain adaptation approaches of the transfer learning approaches.

First, we explored the transfer learning from the bottommost setting through unsupervised domain adaptation. We propose a novel unsupervised domain adaptation approach for object detection. The conventional unsupervised domain adaptation methods can be categorized into feature-level domain adaptation and pixel-level domain adaptation. However, feature-level domain adaptation has the source-biased discriminability problem on the object detection layers, and pixel-level domain adaptation has the imperfect translation problem that does not completely transform source samples from the source domain to the target domain. To solve the source-biased discriminability, we propose Domain Diversification that intentionally causes several distinctive domain shifts from the source domain to enrich the distribution of the labeled data, thereby unbiasing the prediction layers. Moreover, we propose Multi-domain-invariant Representation Learning (MRL) to reduce the domain discrepancies among source domain, target domain, and the diversified domains.

Second, we extend our research area to semi-supervised domain adaptation since the unsupervised cross-domain adaptation setting is far unrealistic for real-world adaptation. The novel setting of the semi-supervised domain adaptation (SSDA) problem shares the challenges with the domain adaptation problem and the semi-supervised learning problem. However, a recent study shows that conventional domain adaptation and semi-supervised learning methods often result in less effective or negative transfer in the SSDA problem. In order to reasonably interpret this observation and address the SSDA problem, we raise the intra-domain discrepancy assumption within the target domain. Then, we present attraction, perturbation, and exploration schemes to solve the semi-supervised domain adaptation problem in the perspective of the intra-domain discrepancy.

Finally, we focus on generalizing a 3D vision task by reducing its dependency on ground-truth through semi-supervised learning. We propose a semi-supervised learning method and a neural network architecture that perform comparable to the supervised MVS methods even though we only used 30 to 40 3D points among dense 3D ground-truth millions of 3D points. To achieve the goal, we divide the reliable and erroneous regions and individually conquer them. We maximize the discriminability of the feature in a self-supervised approach on the reliable regions and propagating the reliable accurate depth predictions to the fundamentally erroneous regions. First, we propose a 3D point consistency loss to enhance the depth accuracy on the non-occluded region. It regresses the back-projected 3D points of the corresponding pixels to actually meet at the same 3D coordinate, so that they can eventually form a correct correspondence in the 3D world. Then, we design propagation approach that update the uncertain depth prediction based on the feature similarity between the nearby pixels.