Learning from computer simulations to tackle real-world problems

Abstract

Deep learning has made great strides for problems that can be learned with direct supervision, in which a large dataset with high-quality annotation is provided (e.g., ImageNet). However, collecting such a large dataset is expensive and time-consuming. The goal of this work is to address this issue by utilizing computer simulation to train a machine learning model. First, we present our novel \emph{Simulated+Unsupervised (S+U) learning} algorithm, which fully leverages the flexibility of data simulators and bidirectional mappings between synthetic and real data. We show that our approach achieves the improved performance on the gaze estimation task, outperforming the prior approach by \cite{shrivastava2016learning}. Next, we introduce our work on building data-driven vehicle collision prediction algorithms. Today’s vehicle collision prediction algorithms are rule-based and have not benefited from the recent developments in deep learning. This is because it is almost impossible to collect a large amount of collision data from the real world. To address this challenge, we collect a large accident dataset using a popular video game named GTA and train end-to-end classification algorithms which identify dangerous objects from a given image. Furthermore, we devise a novel domain adaptation method with which we can further improve the performance of our algorithm in the real-world.