Q($\lambda$) learning based route guidance of overhead hoist transport vehicle in automated material handling system

Abstract

Overhead Hoist Transport (OHT) vehicle is widely used in the automated material handling system (AMHS) of semiconductor wafer fabrication facilities (FAB). Hundreds of OHTs deliver wafer lots on complex tracks, efficient operation of the OHT is one of the key factors of the overall performance of the modern semiconductor FAB. Commonly used routing methods include static Dynamic Programming (DP) approach, periodically updated DP approach, and online DP approach. However, routing based on the existing optimization approach can not effectively respond unexpected events such as delay time due to the OHT loading / unloading or congestion in the track. In this paper, we propose a routing algorithm based on Q ($\lambda$) learning in order to operate the automated transport system efficiently, which avoids congestion and can be applied to actual field. It gathers real-time routing information and measures the expected transport time for each alternative route. The learning-based algorithm is used to determine the next position of the OHT while avoiding congestion in the track. We also verify the performance of the algorithm through simulation experiments. Compared with the practical routing approach, the proposed method showed better performance in terms of the median of average lots delivery time. This study is conducted with one of the biggest global AMHS providers in the semiconductor industries.