Domain-adaptive and robust learning based on adversarial training

Abstract

Although a large number of studies have demonstrated the ability of deep neural networks to solve challenging tasks, it behave abnormally given the target data generated from shifted distribution. For example, the networks trained with data from a specific domain is prone to over-fitting, which makes it difficult to transfer to other domains. Moreover, the trained network is vulnerable to the human-imperceptible adversarial noise, questioning its stability. In this study, two adversarial regularization algorithms are proposed to solve the aforementioned problems. The main contributions of this thesis are as follows. First, we situate the domain adaptation in the context of information theory based on the mutual information between the domain label variable and representations. Based on the findings, we propose the adversarial training algorithm which can exploit the domain-shared information with a single discriminator. Moreover, inspired from the noise distribution experimentally found in the brain, the adversarial training algorithm is proposed to make the networks robust to the adversarial noise.