(A) study on meta-learning-based continual learning framework for scalability of motion prediction model

Abstract

Motion prediction plays a key role in autonomous driving to help the autonomous vehicle to be safe and efficient. When deploying such a motion prediction AI model into the real world, the model must be robust and able to adapt and scale to new environments. To achieve this, accurate position estimation is essential since the motion prediction is based on the past trajectory of surrounding vehicles. The sensor modules of current autonomous driving system are usually equipped with radar and a camera. However, neither of these sensors can provide sufficient performance with regard to the localization of surrounding vehicles due to their limited inherent characteristics. In order to develop an autonomous driving system, more accurate bird’s-eye view localization with regard to surrounding vehicles is necessary. Thus in this study, first we propose a sensor fusion vehicle position estimation framework based on the localization of a vehicle’s rear corner part. The position of the surrounding vehicle is defined by the range data of radar and the angle data which is calculated by localizing the corner part of the vehicle using a camera. Since the vehicle rear corner part is tracked separately, it enables improved robustness with reference to occlusions. That is, surrounding vehicles with occluded views can still be localized. And we also propose a model adaptation process to adapt and scale to new environments smoothly. Since the driving scenarios in the real environment are infinite, the performance of the model inevitably deteriorates when facing new scenarios that have yet to be seen in the training stage. Thus, an adaptation process to the new driving environment is essential. The conventional adaptation process leads to tremendous resources and costs from the data collection, annotation, and training process. Moreover, this problem becomes more crucial in the motion prediction task since collecting motion prediction data is costly and labor-intensive. In this paper, we propose the meta-learning based training framework to increase the adaptability and scalability of the motion prediction model and to reduce the data collection and training efforts. The proposed framework consists of two stages: training and adaptation. In the training stage, the model finds an optimal initialization point that can easily adapt to various driving environments not included in the training dataset based on meta-learning for stable and fast adaptation in new driving patterns and environments. Coping with changing target environments (e.g., driving scenarios from different road types), in the adaptation stage, we continuously update the model parameters learned in the training stage in a way that avoids interference with the previously learned environment. Through this, in updating the model, we propose an optimal adaptation method that prevents the performance of the previously learned environment from deteriorating. Thanks to our training and adaptation method, the proposed framework effectively scales the motion prediction model to new environments while minimizing catastrophic forgetting.