Practical evolutionary reinforcement learning with enhanced sample efficiency

Abstract

We introduce scalable Evolutionary Reinforcement Learning (ERL) Algorithms that combine Evolutionary Algorithm (EA) with Reinforcement Learning (RL) for better sample efficiency. It is widely known that the EA algorithms are less sample efficient than RL algorithms, but they are more stable. Combining two algorithms leads to more sample-efficient performance and stability. In this thesis, we introduce new asynchronous actor and critic update rules for scaling ERL algorithms and apply them to real-world applications where the sample efficiency is more crucial than simulated environments. The selected real-world environment is a camera parameter control task that is difficult to build a simulator. It is shown that the proposed ERL algorithm can achieve higher performance with less samples than conventional RL algorithms in both simulated and real-world environments.