Disentangled representation learning for reference-guided image editing

Abstract

With the recent advances of deep learning especially on Generative Adversarial Networks (GANs), it became possible to modify high-level attributes of an image, or translate between image domains that share content features and differ in style. Current image editing algorithms basically handle relatively general features such as face attribute and style of Van Gogh's paintings. In other words, only common features that a collection of images share can be edited. Image editing at a level of single reference image can show much more various type of conversion. In order to carry out this kind of image editing such as virtual fitting and manga colorization, it is necessary to capture single-image-level features, but general features are too coarse to represent a specialized image. In order to tackle this problem, we introduce a conditional generative model for a reference-based image editing which is controlled by representations of a base image and a reference image. Our proposed model can then take a base image and a reference image as an input and properly combine them so that a desired final image can be generated. In this thesis, we describe the objectives and the architecture of the model and present the image editing results of the model through a test of the replacement of the intermediate representation to a guidance image.