Semantic multi-style transfer using pseudo-supervised learning for anime style transfer

Abstract

Since various objects in anime have their own unique styles, anime style transfer can be seen as an object-to-object multi-style transfer problem. However, the state-of-the-art generative adversarial networks (GAN) for anime style transfer fails to transfer each real-world object to the corresponding anime object style properly. This is because the unsupervised learning cannot provide the semantic mappings between the multi-style objects. In this paper, we propose a new learning framework, called pseudo-supervised learning with a new GAN model, called AnimeGAN. Pseudo-supervised learning utilizes pseudo ground truths for multi-style anime objects so that our AnimeGAN can stably learn the semantic mappings between the real-world and multi-style anime objects. Moreover, we propose a novel single generator network that can embrace the multiple styles of various anime objects. For this, our generator is specifically designed to have three effective processing blocks: densely-connected channel attention block (DCCAB), down-scaling channel attention block (DSCAB), and up-scaling channel attention block (USCAB). Qualitative and quantitative evaluations show that our AnimeGAN generates much more pleasing anime-styled images than the state-of-the-art models.