Study on effective self-learning for aerial vehicles using emulated flight test environment based on magnetic levitation

Abstract

it allows the MAV physical model to learn how to fly by itself, without the risk of damage. The proposed methodology utilizes a magnetic levitation based safety-guaranteed flight test environment and deep reinforcement learning techniques to avoid the current problems in flight control system design procedures, such as the reality gap issue of computational simulations, and the safety issues inherent to real flight testing. The safety-guaranteed flight test environment was achieved using a developed magnetic suspension and balance system (MSBS), which dynamically adjusts magnetic forces interacting with a magnetically levitated MAV. As a result, the MAV can perform either a free flight test for reinforcement learning, or can be constrained for safety. This learning environment was developed to permit safe reinforcement learning of MAV, since trial-and-error based learning typically makes the MAV unstable. In this regard, the MAV can learn to fly itself based on trial-and-error, by interacting with the emulated free flight test environment. Notably, the entire learning process can be conducted without numerical models for both the MAV itself and the flight environment, and the safety of the MAV is guaranteed, even when attempting undesirable actions which might cause the model to become unstable. This approach has unique advantages to permit the effective design of a flight control system, by reducing the modeling errors typical of computational simulations, and preventing the risk of damage in real flight tests. This could eventually enhance the benefits of reinforcement learning for developing advanced flight control systems, which has shown the great potential based on its outstanding control performance with adaptability to mutable dynamics and environments.

This paper presents a novel methodology for designing the flight control system of a micro aerial vehicle (MAV)