Learning task invariance with analogy making

Abstract

Learning compact state representations from high dimensional and noisy observations is the cornerstone of reinforcement learning (RL). However, these representations are often strongly biased toward the current task structure and associated goals, making it hard to generalize to other tasks. Inspired by the human analogy-making process, we propose a novel representation learning framework for learning task-invariant action features in RL. It consists of task and action relevant encoding, hypothetical observation generation, and analogy making between the original and hypothetical observations. Our model introduces an auxiliary objective that maximizes the mutual information between the generated image, and existing labels of codes used to generate the image. Experiments on various challenging RL environments showed that our model helps the RL agent generalize by effectively separating the action features from others. We also interpreted the role of our model as an information-theoretic perspective.