Joint embedding motion in-betweening

Abstract

As general users can easily access deep learning models that automatically generate artwork, technologies that can reflect users’ intentions in more detail are being developed. However, existing research on motion in-betweening primarily supports pose constraints with limited capacity, and no known research focuses on complex joint keyframe constraints. Particularly, transformer-based models for motion in-betweening that have demonstrated strong performance recently employ pose embedding. However, this approach is not ideal for handling each joint independently. This paper proposes a method to modify the structure of the existing transformer-based motion in-betweening model to facilitate motion in-betweening under complex joint keyframe constraints, using joint embedding. It also covers how to improve performance via certain modules and loss functions when using joint embeddings. The experiment was conducted on the LAFAN1 dataset, and it was confirmed that the improved model performed better than the model using pose embedding for some evaluation indicators and tasks. Furthermore, a visualization method was proposed that could display complex joint keyframe constraints using color space. In addition to this, a method for visualizing motion in-betweening when complex joint keyframe constraints are given using color space was proposed, and it was confirmed that the method provided an intuitive view for evaluation of reconstruction and smoothness.