Learning conditional motion distribution for generating human motions in indoor environments

Abstract

In this paper, we present a novel data-driven character pose generation framework for indoor environment interactions. Given the desired root and upper-body pose as control inputs, our framework generates the character’s whole-body poses appropriate for furniture of various shapes and combinations while satisfying the control inputs. The pose generation framework is modeled by learning the conditional motion distribution and the diverse conditions are obtained on the control inputs such as the environment surrounding the character. To allow for generating diverse action types, we propose to model the classifier that automatically classifies the action types of the data frame and to model the prior distribution of noise for the pose generator networks as a mixture of Gaussians. Our framework learns the classifier and the prior distribution in an unsupervised manner by using the Gaussian Mixture Variational Autoencoder (GMVAE). We model the pose generator with two normalizing flow-based models that sequentially generate the upper and lower-body poses. We show that this two-part normalizing flows model increases the accuracy of satisfying the control inputs. In addition, we model the estimator network that predicts foot contact states to use the predicted foot contact states as another condition for pose generator models. We show that the foot contact conditions improve the quality of the generated motion. We demonstrate that our framework can create continuous character motion for indoor scene scenarios. We perform an ablation study to evaluate the advantages of the major components of our framework.