Meta distillation for reinforcement learning

Abstract

As active research of deep reinforcement learning makes it possible to apply reinforcement learning to many high-dimensional environments, the sample efficiency of reinforcement learning has been more important. Learning strategy that utilizes background knowledge from previous tasks to new tasks, such as transfer learning and meta-learning, is one common approach for enhancing sample efficiency. In this work, we propose a meta-learning framework Meta-Distillation for Reinforcement Learning (MDRL) that efficiently transfers expert policies from previous environments to a new policy in an unseen environment. A weighted sum of discrepancies between current policy and expert policies is added to policy update loss, and the weights are determined by a weight network that is meta-trained to help training by considering tasks, training sample, and policy training progress. MDRL succeed to data-efficiently adapt new task when given distribution of environment is scarce and diverse.