Adaptive 3D object tracking and socio-physically acceptable human trajectory prediction using BERT for social navigation

Abstract

Mobile robots have been actively applied to real life, but it is still challenging to implement smooth and safe robot navigation in complex and crowded environments such as shopping malls. Therefore, in this dissertation, we study efficient and safe robot navigation in an environment where robots and humans coexist through adaptive 3D object tracking and socio-physically acceptable human trajectory prediction. First, we propose SPriorSeg, which can rapidly and accurately segment sparse 3D LiDAR point clouds into objects using a light-weighted convolutional auto-encoder and a region-growing algorithm. Second, we propose IMM-MIX to robustly track moving objects in diverse motion patterns with sharp changes by mixing multiple motion models with different velocity representations and complexity. Third, we propose Social-BERT and Social-BERT+ to predict the future movement of pedestrians using BERT. Social-BERT can predict a single socially acceptable future trajectory by considering past movement and social interaction with nearby pedestrians. Social-BERT+ can predict multiple socio-physically acceptable future trajectories in a single situation by applying a generative model. It also utilizes pedestrian trajectories and semantic maps to understand socio-physical interaction. Last, we demonstrate that the proposed methods can be applied to real robot navigation by integrating them into a system that predicts trajectories of surrounding pedestrians in real time from 3D LiDAR point clouds.