Deep reinforcement learning using time-series data for collision avoidance of UAV in high-speed environments

Abstract

With urbanization and acceleration, the demand for mission performance within cities and indoors is increasing. One of the key elements for successful indoor flight is the ability to perform real-time data-based collision avoidance. While optimization-based algorithms and rule-based approaches have been proposed for collision avoidance, they have shown limitations in their applicability to real flight environments and the requirement for users to define all scenarios. Deep reinforcement learning (DRL) is a methodology that uses neural network structures for unmanned aerial vehicles to learn autonomously based on their actions, states, and reward states. Deep reinforcement learning learns to tackle partially observable Markov decision processes without users having to define every scenario. In this study, we propose a deep reinforcement learning algorithm that can be applied in random environments, incorporating sequential depth map data merged into a 2D representation to enhance the visual information the reinforcement learning agent perceives. Additionally, we include sequential linear velocity data to better understand high-speed environments as an additional input to the network. The deep reinforcement learning network utilized Proximal Policy Optimization (PPO). The depth map data is processed through a Convolution Neural Network (CNN) for feature extraction, while the linear velocity information is combined with the image network after flattening, which completes feature extraction through a Multi-Layer Perceptron (MLP) network. The network employed in this study can be operated in an end-to-end environment. Furthermore, the validity of this algorithm was verified through simulations.