Understanding Features on Evolutionary Policy Optimizations

Abstract

We analyze two deep reinforcement learning algorithms, gradient-based policy optimization and evolutionary one, by a number of visualization techniques and supplement experiments. As such techniques, filter visualization and saliency map are used to examine whether meaningful features properly extracted in the two algorithms. In addition to visual analysis, some experiments are devised to enhance the validity of the analysis. We observed that an evolutionary policy optimization tends to make use of the prior knowledge and learn the prior action distribution of the policy by a powerful exploration ability, which a gradient-based algorithm cannot do easily.