Low-thrust spacecraft trajectory and guidance law design using reinforcement learning

Abstract

In this paper, we propose low-thrust spacecraft guidance for multi-revolution orbit transfer using the Soft Actor-Critic (SAC) reinforcement learning (RL) algorithm. Assuming the thrust magnitude is constant, the guidance system of a spacecraft must provide the appropriate direction of the thrust at a given state to reach the desired orbit, satisfying mission requirements. We trained the RL agent that decides the thrust directions over the entire multi-revolution orbit transfer, satisfying the terminal boundary condition and minimizing flight time simultaneously. The new form of the gradient-aided reward function was designed to achieve the fast and accurate training. As a result, the trained agent was able to reach the desired orbit within the error tolerance while minimizing the flight time. Because the proposed approach used a model-free algorithm, this approach has a potential to be used under other environments. The robustness of the proposed algorithm to dynamics uncertainty was tested via Monte-Carlo simulations because the training environment and the real environment would be different. The trained agent was robust to the certain level of position and velocity error, and the results were different upon orbit transfer scenarios.