Conditional person image synthesis: full-body virtual try-on (VITON) reflecting wearing styles

Abstract

We propose a conditional generative neural network that synthesizes a realistic person image reflecting the input image of a person, multiple garments, and wearing styles. Firstly, we construct a benchmark dataset Fashion-TB with paring information between a person and multiple clothes that the person is wearing. We propose a method to properly embed input conditions (i.e., person, clothes, and wearing styles) and reflect the latent feature to the generative neural network so that the network effectively generates a person image under the given conditions. In particular, we propose a single-stage network trained by end-to-end learning while most existing studies consist of multiple stages to leverage performance. In spite of the simple architecture with fewer parameters, our proposed network can effectively synthesize VITON images. Additionally, we introduce two methods called Wearing-guide Scheme and Wearing Style Transfer which control the wearing style of the synthesized image using a user-defined binary mask and an example image, respectively. We evaluate the proposed dataset and methods compared to stateof-the-art methods. We can verify that existing methods can be trained using the proposed dataset, and our proposed methods can synthesize a more realistic person image. Also, methods to control wearing style are validated to alter the wearing style of the output image, and they can be utilized in two different user scenarios.