Segment-aware motion model for image animation

Abstract

Motion Transfer Model animates an image by capturing motion from a driving video. In most motion transfer models, unsupervised learning is commonly employed, which neglects the consideration of segment information in the images and results in motion estimation without segment awareness. This negatively impacts each module of the motion transfer model and ultimately affects the quality of the generated videos. In this paper, we analyze the impact on each motion transfer module when segment information is not taken into account and propose \emph{Segment-Aware Notion Transfer Model} that utilizes segmentation maps for motion transfer. We validate our approach on benchmark dataset and observe improvements in the image quality of the genereated video, both on quantitatively and qualitatively.