Rich360: Optimized Spherical Representation from Structured Panoramic Camera Arrays

Abstract

This paper presents Rich360, a novel system for creating and viewing a 360 degrees panoramic video obtained from multiple cameras placed on a structured rig. Rich360 provides an as-rich-as-possible 360 degrees viewing experience by effectively resolving two issues that occur in the existing pipeline. First, a deformable spherical projection surface is utilized to minimize the parallax from multiple cameras. The surface is deformed spatio-temporally according to the depth constraints estimated from the overlapping video regions. This enables fast and efficient parallax-free stitching independent of the number of views. Next, a non-uniform spherical ray sampling is performed. The density of the sampling varies depending on the importance of the image region. Finally, for interactive viewing, the non-uniformly sampled video is mapped onto a uniform viewing sphere using a UV map. This approach can preserve the richness of the input videos when the resolution of the final 360 degrees panoramic video is smaller than the overall resolution of the input videos, which is the case for most 360 degrees panoramic videos. We show various results from Rich360 to demonstrate the richness of the output video and the advancement in the stitching results.