Sequential image-based 3D object detection with depth-offset guided location refinement

Abstract

Detecting objects in the surrounding environment is essential for autonomous driving to predict the surrounding vehicle's future motion and plan a route. Deep learning has become an active research tool for object detection tasks due to its ability to extract increasingly better features. The image-based object detection network mimics the human cognitive process and detects objects based on rich texture information. However, since the image-based 3D object detector is limited to 2D image plan data, it is impossible to infer 3D depth information accurately. Most image-based 3D object detectors do not utilize sequential information, even though the object's historical location and movement are significant for predicting the next frame in the human cognitive process. In this paper, we propose a new method to reduce object 3D location inference error and improve 3D object detection accuracy by considering the displacement of an object in sequential images. First, the proposed network is trained with a pair of sequential images, and the network predicts the uncertainty of the object's depth as an additional output. Second, the final 3D location is refined based on the object's depth in the two sequential images, the displacement in the depth direction, and the uncertainty for each. The experimental results demonstrate that our proposed method reduces average translation error and improves the detection accuracy by properly propagating the object's information in the previous frame to detect the object in the current frame. The method proposed in this paper is expected to play a vital role as an advanced technology to improve the accuracy of 3D object detection.