Localizing Human Keypoints Beyond the Bounding Box

Abstract

Since human pose is one of the most effective and popular sources for understanding human in various applications, there have been numerous researches on detecting keypoints of human body from the image source. However, when a human body is shown partially in the source image, estimation range is also restricted causing performance degradation in locating keypoints of human body. In this paper, we propose ‘Position Puzzle’ network and augmentation to leverage the performance of detecting keypoints including those outside the bounding box. Specifically, Position Puzzle Network expands the spatial range of keypoint localization by refining the position and the scale of the target’s bounding box, and Position Puzzle Augmentation improves the performance of keypoint detector using the partial image in training. We prepare data by cropping COCO dataset and utilize them in training and evaluation. Under the prepared dataset, the proposed method enhances the performance of baseline network up to 37.6% and 30.6% in mAP and mAR, respectively, and effectively localizes keypoints positioned not only inside but also outside the bounding box. We also verify that the proposed method can localize keypoints beyond the bounding box in the original COCO dataset.