Representation learning for visual control with world models

Abstract

Model-based approaches, which predict future consequences of potential actions and make decisions based on these predictions, hold substantial potential for efficiently learning to achieve target tasks. The capability of model-based agents relies on the accuracy of planning, but learning a world model with accurate planning capability is often difficult and costly. This dissertation argues that improving the representation learning from high-dimensional visual observations would enable us to efficiently learn world models and endow agents with the planning capability. First, we show how pre-training representations from diverse, action-free videos can accelerate world model learning on unseen environments, thereby reducing the number of samples required for solving the newly encountered tasks. Second, we present a new model-based framework that decouples visual representation learning and dynamics learning, along with a self-supervised learning approach that adapts a recently-developed masked autoencoding approach to be better suited for visual control. Finally, we extend our framework to a practical robot learning scenario that utilizes multiple cameras, by introducing a novel representation learning method that reconstructs masked viewpoints to learn cross-view information. The approaches we present in this thesis demonstrate strong empirical results in both simulated and real-world benchmarks, highlighting the importance of learning succinct visual representations for world model learning.