Design of an aerial combat algorithm based on basic fighter maneuvers and reinforcement learning

Abstract

Unmanned combat aerial vehicles are currently restricted to reconnaissance and bombing missions. However, they will be expected to assist human pilots, or perform autonomous aerial combat in the near future. Various research treating unmanned aerial combat have limitations associated with the overly simplifying assumptions, computational complexity, and limited scalability. In this study, a novel autonomous aerial combat algorithm with high-performance and real-time calculations is proposed based on basic fighter maneuvers for gun-based engagement within visual range. From the perspective of energy-maneuvering, which is the core concept of basic fighter maneuvers, flight envelopes and the maximum maneuvering point of a fighter jet model is analyzed through the velocity-load factor and energy-maneuver diagrams. To assess the combat situations, a score function and matrix are designed based on the combat geometry, enabling cooperative maneuvers and role assignment. By performing gun modeling and target prediction based on the maneuver plane, the impact point for gun shooting is obtained. Furthermore, deep Q-network, one of reinforcement learning method, is adopted to learn the new novel tactics for complex two-on-two air combat against pre-designed BFM-based combat algorithm. Two different types of simulation environments are organized to validate the combat performance: pseudo 6 degree-of-freedom model based MATLAB environment, and X-Plane with non-linear high fidelity model and MATLAB/Simulink combined real-time virtual combat environment. The performance of the designed algorithm was validated through not only Monte-Carlo simulations with various random distributed initial conditions, but also engagements with another algorithm for comparison, and evaluation sessions by an Air Force combat instructor in a real-time simulation environment using virtual reality goggles and X-Plane.