Multi-agent reinforcement learning with a graph neural network for OHT traffic control

Abstract

A large-scale engineering system is typically composed of numerous agents conducting specified tasks to achieve system-level goals. It is generally challenging to derive a control policy that maps the system level state to jointly control inputs for all the subsystems due to the difficulties in modeling interactions among agents and coordinating them efficiently toward achieving their goals, especially when there is no prescribed model describing the dynamics of the target system. In this study, we propose an efficient multi-agent reinforcement learning (MARL) algorithm with a graph neural network (GNN) as a base building block to model the interactions among agents and the decentralized policies for every agent. Notably, we use a graph structure to represent the relationships of agents in a system and message passing strategies to coordinate them implicitly in achieving the cooperative goals

we call this method “Graph-MARL.” We implement the proposed GNN-based MARL approach to mitigate the congestion of overhead hoist transports (OHTs) that perform tasks of handling material in a semiconductor FAB. Specifically, the proposed method controls the flows of 48 merging intersections in the target AMHS rail given a current state of the system represented as a graph. Simulation studies demonstrate that the proposed method dramatically reduces the congestion level and improves the productivity of the OHT systems, especially when there is a large number of OHTs densely working in a FAB.