Purpose-built sensor fusion for autonomous vehicles

Abstract

This dissertation proposes a purpose-built sensor fusion, constructing sensor system for autonomous vehicles and selectively fusing them based on purposes. The components and implementation details of an autonomous vehicle are introduced in order to show the process of transforming a mass-produced car into a self-driving car. RGB cameras, near-infrared (NIR) cameras, an inertial measurement device, a GNSS, LiDARs, and a vehicle control area network (CAN) grabber are selectively fused to collect two driving datasets. Specifically, a multi-modal depth dataset for changing environments and a large-scale driving dataset are proposed. Adaptive cost volume fusion network for depth estimation is verified from the proposed multi-modal depth dataset. A lightweight depth completion network with local similarity-preserving knowledge distillation is proposed and verified. Application of the proposed system are introduced. Representation learning method from driving scenes is performed from images with vehicle motion information. Proposed system is utilized to verify the sensor fusion based methods for estimating vehicle dynamics from sensors on the actual road. Dead-reckoning in GNSS-denied environments is performed by fusing the lane information from front camera and the response of the inertial measurement unit, with verification of the performance by comparing the results between the proposed method and those of previous works.