Methodology for heat integration and reinforcement learning-based energy management of liquid hydrogen-powered hybrid ship propulsion system

Abstract

This study proposes heat integration and deep reinforcement learning-based energy management methodologies for efficient operation of a hybrid ship propulsion system (HSPS) utilizing liquid hydrogen (LH$_2$) as fuel and analyzes their effectiveness. The targeted LH$_2$-HSPS consists of a fuel gas supply system (FGSS), polymer electrolyte membrane fuel cell (PEMFC), and lithium-ion battery system. A 2 MW-class platform supply vessel (PSV), exhibiting significant load fluctuations during operation, is selected as the target ship for application of the LH$_2$-HSPS. Specifications of the LH$_2$-HSPS are determined based on operation scenarios of the PSV, and the dynamic model is developed accordingly. Validity of the developed model is confirmed during the validation phase, and using the validated model, design and operational feasibility of the LH$_2$-HSPS are further assessed for various operational strategies. Subsequently, to enhance the energy efficiency of the LH$_2$-HSPS, a proposal is made to integrate an ethylene glycol/water mixture-based thermal management system of the LH$_2$ FGSS and battery system. The dynamic model is then utilized to quantitatively analyze effects of heat integration. Additionally, the proposed methodology's validity is confirmed by investigating temperature changes in the battery system due to heat integration. Finally, a deep reinforcement learning (DRL)-based optimal energy management algorithm applicable to the energy management system (EMS) of the LH$_2$-HSPS is suggested. An objective function of the energy management problem considers hydrogen and equivalent fuel consumption, and performance degradation of PEMFC and lithium-ion battery system. The DRL-EMS is compared with dynamic programming and sequential quadratic programming algorithm to evaluate the global and real-time optimization performance. Furthermore, the optimal energy management performance of the DRL-EMS is assessed for operational scenarios not used in agent training. An analysis of operational strategies is conducted based on the energy management results, considering various hydrogen fuel prices and capacities of the lithium-ion battery system.