A Tensor Voting Approach for Multi-View 3D Scene Flow Estimation and Refinement

Abstract

We introduce a framework to estimate and refi ne 3D scene flow which connects 3D structures of a scene across diff erent frames. In contrast to previous approaches which compute 3D scene flow that connects depth maps from a stereo image sequence or from a depth camera, our approach takes advantage of full 3D reconstruction which computes the 3D scene flow that connects 3D point clouds from multi-view stereo system. Our approach uses a standard multi-view stereo and optical flow algorithm to compute the initial 3D scene flow. A unique two-stage re-fi nement process regularizes the scene flow direction and magnitude sequentially. The scene flow direction is refi ned by utilizing 3D neighbor smoothness de fined by tensor voting. The magnitude of the scene flow is re fined by connecting the implicit surfaces across the consecutive 3D point clouds. Our estimated scene flow is temporally consistent. Our approach is efficient, model free, and it is effective in error corrections and outlier rejections. We tested our approach on both synthetic and real-world datasets. Our experimental results show that our approach out-performs previous algorithms quantitatively on synthetic dataset, and it improves the reconstructed 3D model from the refi ned 3D point cloud in real-world dataset.