Model ensemble-based intrinsic reward for sparse reward reinforcement learning

Abstract

In this paper, a new intrinsic reward generation method is proposed based on an ensemble of dynamics models for sparse-reward reinforcement learning. In the proposed method, the mixture of multiple dynamics models is used to approximate the true unknown transition probability and the intrinsic reward is designed as the minimum of the surprise seen from each dynamics model to the mixture of the dynamics models. Then, a working algorithm is constructed by combining the proposed intrinsic reward generation method with PPO. Numerical results show that the proposed model ensemble-based intrinsic reward generation method outperforms the previous intrinsic reward generation method based on a single dynamics model.