Estimation of depth and 3D motion parameter of moving object with multiple stereo images

Abstract

In this thesis, a use of stereo image sequences is considered to estimate both three-dimensional (3D) motion and depth of moving point features. In practice, a number of problems make it difficult to find correct match between stereo pairs and successive image frames. This thesis attacks the two ambiguity problems : stereo ambiguity resulted from multiple stereo matches, motion ambiguity resulted from multiple motion matches. First, the problem is formulated as optimization of a cost function, accumulated sum of squared differences(SSD) computed from stereo motion sequences. It is assumed that the 3D motion is pure translational within any image window the SSD is computed. The use of multiple stereo images has two advantages as follows. One is that the two-dimensional(2D) feature points can be tracked well if the images are sampled so frequently that visual motions in 2D image plane are small, thus we can easily check temporal consistency to disambiguate multiple stereo matches. The other is a capability of improving achievable precision of the estimates from data redundancy. The unique condition is derived under which 3D motion and depth ambiguities can be resolved with stereo motion sequences. Once the ambiguities are resolved, the 3D motion parameters and depth are estimated by least squares technique. By analyzing the statistical characteristics of the cost function, it is shown that precision of the estimates can be improved from data redundancy. An optimization with forgetting factor is proposed to compensate the effect of rotation and time-varying 3D velocity. Second, a recursive estimation method by Kalman filter approach is presented to reduce computational load and memory space for saving images until resolving the multiple matches. In particular, discarding methods of multiple stereo and motion matches are presented because the multiple matches result in computational explosion. Virtual 3D points are generated from all ambiguous stereo and motion ma...