Video Inference for Human Mesh Recovery with Vision Transformer

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 40
  • Download : 0
Human Mesh Recovery (HMR) from an image is a challenging problem because of the inherent ambiguity of the task. Existing HMR methods utilized either temporal information or kinematic relationships to achieve higher accuracy, but there is no method using both. Hence, we propose 'Video Inference for Human Mesh Recovery with Vision Transformer (HMR-ViT)' that can take into account both temporal and kinematic information. In HMR-ViT, a Temporal-kinematic Feature Image is constructed using feature vectors obtained from video frames by an image encoder. When generating the feature image, we use a Channel Rearranging Matrix (CRM) so that similar kinematic features could be located spatially close together. The feature image is then further encoded using Vision Transformer, and the SMPL pose and shape parameters are finally inferred using a regression network. Extensive evaluation on the 3DPW and Human3.6M datasets indicates that our method achieves a competitive performance in HMR.
Publisher
Institute of Electrical and Electronics Engineers Inc.
Issue Date
2023-01
Language
English
Citation

17th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2023

DOI
10.1109/FG57933.2023.10042731
URI
http://hdl.handle.net/10203/305995
Appears in Collection
EE-Conference Papers(학술회의논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0