LiDAR-centric multimodal fusion for 3D semantic segmentation in autonomous driving

Abstract

3D point cloud semantic segmentation in autonomous driving aims to extract useful information from 3D sensors and assist autonomous vehicles in performing downstream tasks based on the predicted class labels of point clouds. Thanks to advances in sensor technology and the improvement of deep learning techniques, this area has garnered increasing attention in recent years. However, outdoor 3D point clouds pose a significant challenge for deep learning approaches due to their unordered, sparse and irregular nature. In this dissertation, we investigate point cloud data preprocessing, feature representations, deep learning framework design and multimodal fusion to study single-modal and multimodal segmentation approaches. Specifically, we propose: 1) a rule-based ground segmentation approach that utilizes a bird’s-eye view (BEV) log-polar grid map and rule-based ground model algorithms for binary ground segmentation

2) an efficient deep learning framework for ground segmentation called SectorGSNet, which employs a BEV log-polar transform and a lightweight network

and 3) a multimodal fusion semantic segmentation approach that leverages complementary data from LiDAR and RGB cameras for range-view based semantic segmentation.