Generalizable robust deep learning via adversarial pruning and meta-noise generation

Abstract

Despite the remarkable performance of deep neural networks on various computer vision tasks, they are known to be susceptible to adversarial perturbations, which makes it challenging to deploy them in real-world safety-critical applications. In this thesis, we conjecture that the leading cause of the adversarial vulnerability is the distortion in the latent feature space and provide methods to suppress them effectively. We propose a Bayesian framework to prune features with high vulnerability to reduce vulnerability and loss on adversarial samples. We validate our Adversarial Neural Pruning with Vulnerability Suppression (ANP-VS) method on multiple benchmark datasets. It obtains state-of-the-art adversarial robustness and improves the performance on clean examples, using only a fraction of the parameters used by the complete network. We further propose a novel meta-learning framework that explicitly learns to generate noise to improve the model’s robustness against multiple types of attacks. Its key component is Meta Noise Generator (MNG) that outputs optimal noise to stochastically perturb a given sample, such that it helps lower the error on diverse adversarial perturbations. We validate the robustness of models trained by our scheme on various datasets and against a wide variety of perturbations, demonstrating that it significantly outperforms the baselines across multiple perturbations with a marginal computational cost.