Algorithms for efficient offline reinforcement learning

Abstract

Offline reinforcement learning (RL) aims to learn without additional interaction with the environment, based on the pre-collected dataset. It has recently gathered attention due to its promise for real-world applications. Unlike online RL where the agent's predictions can be further corrected through additional interactions, offline RL requires robust policy improvement under the potentially incorrect predictions. To do this, it is necessary to accurately measure the uncertainty of the implicitly or explicitly constructed environment model, and design an algorithm that can find a trade-off between the potential policy performance and the uncertainty in policy evaluation. In this thesis, we study offline RL algorithms about (1) finding a good trade-off using a validation split and (2) learning model that is more robust especially for offline RL.