Generating 3D human texture from a single image with sampling and refinement

Abstract

We propose a novel method for generating a texture map for a 3D human model from a single image. Generating the texture map for a 3D human model from a single image is challenging because the image provides only partial information. To generate a plausible texture map, occluded textural patterns, which are invisible in the source image, need to be synthesized with relevance to the visible region of the image. Moreover, the generated texture map should be semantically aligned with the UV space of the template mesh. To generate a complete texture map from a single image while overcoming the aforementioned challenges, we propose a novel texture synthesis method that incorporates networks for sampling and refinement. The sampler network fills in the occluded regions of the source image in texture space using the information from the visible region. The refinement network refines and adjusts the sampled texture to create a detailed and aligned texture in the UV space of the template mesh. We conducted experiments to show that our method outperforms previous methods qualitatively and quantitatively.