Learning critic network using priority in proximal policy optimization algorithm

Abstract

Proximal policy optimization algorithm (PPO) is one of the representative methods of policy-based reinforcement learning using actor-critic networks. It has been used as a baseline in various works on reinforcement learning. In this thesis, we newly consider the priorities of samples in the learning process of the critic neural network of the original proximal policy optimization algorithm. With the help of priorities, we accelerate learning the value function faster which can help learning of the actor neural network. We use two different prioritization methods: one using a time difference error as in the prioritized experience replay of deep Q networks and the other one using the Gaussian process regression.