Ranking and selection framework for efficient reverse simulation

Abstract

A stochastic simulation is a powerful tool for evaluating the performance of man-made complex systems, where the performance is evaluated by independent repeating simulations. The reverse simulation means finding optimal system designs satisfying given performance requirements of the system among various design alternatives, based on the stochastic simulation. If the number of design alternatives is small and limited, the ranking and selection (R&S) in statistics is an efficient approach for the reverse simulation. When computing resources available for repeating simulations are limited, the R&S aims to find optimal design accurately by distributing the limited resources to design alternatives effectively. Recently, with increasing complexity of the systems, computing resources per simulation run also increases rapidly

thus, a problem of increasing the efficiency of R&S for selecting optimal design accurately with minimized resources has been drawing attention. For this, three frameworks have been proposed: the optimal computing budget allocation (OCBA), the indifference-zone (IZ), and the expected opportunity cost (EVI), and many efficient algorithms for various R&S problems has been developed based on these frameworks. However, the previous frameworks have a disadvantage in that the efficiency decreases when simulation model has large stochastic noise. This dissertation proposes an efficient R&S framework for efficient reverse simulation of the stochastic simulation model. To effectively allocate limited simulation resources, this dissertation defines a measurement named “uncertainty” based on the p-values of statistical hypothesis test and suggests a heuristic allocation policy of additional simulation resources for maximizing the accuracy based on the uncertainty. To use the limited resources more efficiently, the UE framework allocates the additional simulation resources step by step using the defined uncertainty and the policy, based on the sequential procedure. While the previous frameworks focus on only the sample mean and sample variance when allocating resources, the proposed framework considers the standard error additionally. This further consideration improves the efficiency of the framework in noisy situations by reducing wasted resources due to a poor value of sample mean. In this dissertation, based on the proposed framework, we propose efficient algorithms for solving various R&S problems. We demonstrate its improved efficiency and high robustness to noise by comparative experiments with the existing algorithms developed based on the previous frameworks. We also introduce several case studies in which the reverse simulation of the real system is efficiently performed through the proposed algorithm.