Deep learning based vehicle position and orientation estimation via inverse perspective mapping image

Abstract

The architecture of the autonomous vehicle can be roughly divided into the perception that recognizes the surrounding environment, prediction of the behavior of the surrounding vehicles, decision-making to generate the path of the ego vehicle, and motion control. Since the perception is performed at the first stage, the performance of the prediction and decision-making is greatly affected by the performance of the perception. Therefore, it is essential to improve the perception performance to accurately estimate the distance, velocity, and direction of surrounding vehicles for advanced autonomous driving technology. The latest perception algorithm has been commonly developed by using LIDAR sensors that measure the distance from the target by emitting a laser to accurately detect the position of the object. However, LIDAR sensor is one of the obstacles to commercializing autonomous vehicle technology due to high price. Although the vehicle detection using an inexpensive monocular camera has been studied, previous researches mainly perform detection in the image coordinate system. It is difficult to measure the distance of the surrounding vehicles or to predict the behavior since the information obtained with the pixel units in the image coordinate system does not indicate the distance to the surrounding vehicles. In addition, the orientation of the surrounding vehicles is also an important factor in order to predict the behavior of the surrounding vehicles, but the previous studies using the camera cannot deal with the direction. Therefore, the position and orientation information of surrounding vehicles in the world coordinate system is required to accurately predict the behavior of the surrounding vehicles in the autonomous vehicle.

In order to solve the above problem, this thesis proposes the method to detect the position and orientation and measure relative distance of the surrounding vehicle using deep learning network on the bird's-eye image projected by inverse perspective mapping (IPM). The proposed algorithm uses the following processes to efficiently detect the position and direction of surrounding vehicles using the monocular camera. The front view image captured by the monocular camera in the pixel image coordinate system is stabilized by using the extrinsic parameter or camera and inertial measurement unit (IMU) information to correct the pitch and roll motion of the vehicle. The stabilized front view image is projected onto a bird's-eye view image so that it is parallel to the road surface of the world coordinate system using IPM. Then deep learning network takes the bird's-eye view image as input and outputs the position, size, and orientation of the vehicle. Finally, detected vehicle on the bird’s-eye view image with pixel unit is converted to meter unit to estimate the relative distance.

The proposed algorithm was evaluated on the KITTI raw dataset and showed average precision of 62% and 19% at a 0.5 and 0.7 intersection over union (IoU), respectively. In order to evaluate the performance of the algorithm in the driving situation, the distance estimation accuracy in the world coordinate system was evaluated on KITTI raw dataset and showed a root mean square error (RMSE) of 0.074 meters in the lateral direction and 0.81 meters in the longitudinal direction with the operating speed of 0.024 seconds (40 FPS).