Supervisory control based reinforcement learning for scheduling semiconductor manufacturing cluster tools

Abstract

Cluster tools are widely used in most semiconductor manufacturing processes such as etching, deposition and metallization. Scheduling of cluster tool is a decision problem of the behavior, timing, and target modules in the tool, and it greatly effects on the productivity of the tool. To improve the performance of semiconductors, manufacturers has shrunk the wafer circuit width dramatically, which has led to a significant increase in the complexity of tool scheduling problem. Furthermore, considering variability such as variation in process time and disruptive events such as chamber failure increases the complexity of scheduling further more. Scheduling methods for specific tool structures and operational constraints have been studied, but no studies have been conducted considering complex operational constraints. Mathematical optimization methods such as mixed integer programming are unsolvable due to their computational complexity. In this work, we present a supervisory control-based reinforcement learning approach that enables near-optimal scheduling even in situations involving complex constraints on cluster tools. First, we propose an algorithm for logical control and performance optimization for cluster tools without variability and disruptive events. Second, we extend it to tools which includes variability and disruptive events. Then, interpretable rules are extracted and analyzed from optimized scheduling policy. Through this study, near-optimal scheduling is derived for complex problems that cannot be solved using the previous method, and interpretable rules are extracted. Numerical experiments demonstrate that proposed algorithm outperforms the state-of-the-art cyclic scheduling methods and achieves near-optimal performance even on problems where the scheduling method is unknown yet.