3D video stabilization using optimal trajectory of point feature and its application to a humanoid robot

Abstract

This paper deals with video stabilization by using image sequences captured by stereo cameras mounted on unstable mobile platforms. The video stabilization has been an important topic in image processing and computer vision, and has been approached in many ways because it can be applied to many practical applications in the field of consumer electronics such as digital camcorder, computer vision, and robotics. When biped humanoid robots move, an image sequence acquired from the head-mounted camera can be shaky due to the impact of walking or running. This unstable image sequence places a cognitive burden on the remote operator who controls the robot. Moreover, shaky image sequences can degrade the performance of vision processing such as visual detection, tracking and recognition of human or objects. 2D video stabilization that uses a 2D camera motion model works well if scenes are far from the camera mounted on the mobile platform or can be modeled as a plane. However, because most of the robots operate in a real 3D space, these 2D video stabilization approaches may fail to stabilize unstable videos. Several 3D video stabilization approaches using a 3D camera motion model, which has three rotational and three translational camera motions in a 3D space, have been recently reported in the computer vision community. And very surprising results overcoming the limitations of 2D video stabilization have been presented. However, these 3D video stabilization approaches cannot be directly applied to a humanoid robot vision systems, because they use an off-line batch optimization method and cannot process in real-time. Furthermore, straightforward 3D video stabilization methods, which depend on structure-from-motion to estimate global camera motion, might fail in the long run because accumulative error in the global motion estimation step is inevitable with the elapse of time. This paper proposes new online 3D video stabilization method for a humanoid robot without ex...