Cost effective design of wireless charging transportation system based on reinforcement learning

Abstract

This thesis proposes a wireless charging transportation system model and optimization algorithm based on reinforcement learning. The design of conventional electric vehicles (EVs) is affected by numerous limitations, such as a short travel distance and long charging time. As one of the first wireless charging transportation systems, the Online Electric Vehicle (OLEV) was developed to overcome the charging limitations of the current generation of EVs. Using wireless charging, an electric vehicle receives power wirelessly from power cable embedded in the road. In order to successfully support the operation of wireless charging transportation system as public transportation, it is necessary to develop an accurate model of wireless charging transportation system and optimization algorithm. Therefore, we propose a system model reflecting actual traffic environment and an optimal design algorithm based on reinforcement learning. First, a model and algorithm for the optimal design of a wireless charging electric bus system is proposed. The model is built using a Markov decision process and is used to verify the optimal number of power cable segments, as well as optimal pickup capacity and battery capacity. Based on the Google Maps API and Google Transit API, traffic environment of M1 bus line in New York City is composed to reflect the diverse traffic environment. Using reinforcement learning, the optimization problem of a wireless charging electric bus system in a diverse traffic environment is then solved. The numerical results show that the proposed algorithm maximizes average reward and minimizes total cost. We show the effectiveness of the proposed algorithm compared to the exact solution via mixed integer programming (MIP). Second, we propose a model and algorithm for optimal design of a wireless charging tram system to minimize the operation cost. The Manchester Metrolink tram network have been built using the Google Maps API and the Google Transit API to extend single route environment to multiple entry and multiple destination environment. Then, we found the optimal battery capacity and pickup capacity value using decentralized multi-agent reinforcement learning algorithm Numerical results show that the proposed algorithm minimizes the cost compared to the single agent reinforcement learning algorithm and presented faster convergence compared to the dynamic programming algorithm. Third, we propose a bi-level reinforcement learning algorithm to minimize the operation cost of wireless charging electric bus system. The stochastic traffic environment is modeled to reflect the traffic condition wireless charging electric bus experiences, such as misalignment of wireless charging module and passenger demand variation according to operation time. Through simulation, we show that the convergence rate of proposed algorithm greatly is improved compared to conventional Q-learning algorithm and output of proposed algorithm is cross checked using mixed integer programming (MIP) algorithm.