3D hand motion estimation from a single motion blurred RGB image

Abstract

Hand Mesh Estimator (HME) has been studied and shown to have high performance. However, although previous HMEs have shown high performance for sharp hand images, their performance is poor for motion blur due to fast movements. Therefore, it is necessary to study HMEs that are robust to motion blur. The main problem in the research of motion blur robust HME is the lack of datasets that provide real motion blur images. Therefore, in this study, we constructed a dataset consisting of real motion blur images. The annotation of the dataset was achieved by using an Adaptive Graph Neural Network (A-GCN) based network, which was trained with a sharp image dataset and a multiview camera environment. Furthermore, a Motion Blur-to-Motion Network (MBMNet) with a channel attention module, a Convolutional LSTM (ConvLSTM), and an A-GCN HME is built and trained with the dataset to predict successive hand motions from a single motion blur image. We show how motion blur can be used as useful information instead of noise.