Image-to-Image translation with object attention

Abstract

Image-to-image translation is an important problem in computer vision, in both theoretically and practically. Recently, image-to-image translation have shown a remarkable success, even for the case where the domain pair data is not provided. However, previous approaches have several limitations, that they mistranslate objects, unable to control which object to translate, and fail to modify the shape of objects. To address this problem, we propose a method that utilize the attention of target objects for imageto-image translation. First, we propose a novel neural network architecture composed of the attention network and the translation network. Second, we propose a novel loss function that promotes the prior knowledge of attention. In addition, we propose two novel modifications and data augmentation scheme to improve the attention. As a result, our proposed method not only reduced the previous limitations of translation algorithms, but also improved the performance of previous attention algorithms. We present the experiment results in both qualitatively and quantitatively.