Video Inference for Human Mesh Recovery with Vision Transformer

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 40
  • Download : 0
Human Mesh Recovery (HMR) from an image is a challenging problem because of the inherent ambiguity of the task. Existing HMR methods utilized either temporal information or kinematic relationships to achieve higher accuracy, but there is no method using both. Hence, we propose 'Video Inference for Human Mesh Recovery with Vision Transformer (HMR-ViT)' that can take into account both temporal and kinematic information. In HMR-ViT, a Temporal-kinematic Feature Image is constructed using feature vectors obtained from video frames by an image encoder. When generating the feature image, we use a Channel Rearranging Matrix (CRM) so that similar kinematic features could be located spatially close together. The feature image is then further encoded using Vision Transformer, and the SMPL pose and shape parameters are finally inferred using a regression network. Extensive evaluation on the 3DPW and Human3.6M datasets indicates that our method achieves a competitive performance in HMR.
Institute of Electrical and Electronics Engineers Inc.
Issue Date

17th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2023

Appears in Collection
EE-Conference Papers(학술회의논문)
Files in This Item
There are no files associated with this item.


  • mendeley


rss_1.0 rss_2.0 atom_1.0