Practical Reinforcement Learning for Adaptive Photolithography Scheduler in Mass Production

Abstract

This work introduces a practical reinforcement learning (RL) techniques to address the complex scheduling challenges in producing Active Matrix Organic Light Emitting Diode displays. Specifically, we focus on autonomous optimization of the photolithography process, a critical bottleneck in the fabrication. This provides an outperforming scheduling method compared with the existing rule-based approach which requires diverse rules and engineer experience on adapting dynamic environments. Our purposing RL network was designed to make effective schedules aligning with layered structures of the planning and scheduling modules for mass production. In the training phase, historical production data is utilized to create a representative discrete event simulation environment. The RL agent, based on the Deep Q-Network, undergoes episodic training to learn optimal scheduling policies. To ensure safe and reliable scheduling decisions, we further introduce action filters and parallel competing schedulers. The performance of RL-based Scheduler (RLS) is compared to the Rule-Based Scheduler (RBS) over actual fabrication in a year-long period. Based on key performance indicators, we validate the RLS outperforms the RBS, with a remarkable improvement in step target matching, reduced setup times, and enhanced lot assignments. This work also paves a way for the gradual integration of AI-based algorithms into smart manufacturing practices.