End-to-end control of USV swarm using graph-centric multi-agent reinforcement learning

Abstract

In this paper, we study an algorithm to derive the decentralized and cooperative control strategy for the unmanned surface vehicles (USVs) swarm using graph-centric multi-agent reinforcement learning (MARL). Our model first expresses the mission situation using a graph considering the various sensor ranges. Next, each USV agent encodes observed information into localized embedding and then derives coordinated action through communication with the surrounding agent. Also, We make each agent's policy to maximize the team reward for deriving a cooperative policy. Using the USV combat simulator, we have shown that it outperforms conventional heuristic-based defensive strategies in the training scenarios. In addition, empirically, we showed that proposed model could derive a scalable control strategy through experiments in the unseen scenario.