ImaGAN: Unsupervised Training of Conditional Joint CycleGAN for Transferring Style with Core Structures in Content Preserved

Abstract

This paper considers conditional image generation that merges the structure of one object with the style of another. In short, the style of an image has been substituted or replaced by the style of another image. An architecture for extracting the structure of one image and another architecture for merging the extracted structure and the style of another image is considered. The proposed ImaGAN architecture consists of two networks: (1) style removal network R that removes style information and leaves only the edge information and (2) the generation network G that fills the extracted structure with the style of another image. This architecture allows various pairing of style and structure which would not have been possible with image-to-image translation method. This architecture incurs minimal classification error compared prior style transfer and domain transfer algorithms. Experimental result using edges2handbags and edges2shoes dataset reveal that ImaGAN can transfer the style of one image to another without distorting the target structure.