Novel view synthesis with multiple 360 images for large-scale 6-DOF virtual reality system

Abstract

We present a novel view synthesis method that allows users to experience a large-scale Six-Degree-of-Freedom (6-DOF) virtual environment. Our main contributions are the construction of a large-scale 6-DOF virtual environment using multiple 360 images as well as synthesis of a scene from novel viewpoints. Novel view synthesis from a single 360 image can give free viewpoint experience with full 6-DOF of head motion to players, but the moveable space is limited within a context of the image. We propose a novel view synthesis process that references multiple 360 images via reconstructing a large-scale real world based virtual data map and perform a weighted blending for interpolating multiple novel view images. Our results show that our approach provides a wider area of virtual environment as well as a smooth transition between each reference 360 images.