Enhanced interactive 360° viewing via automatic guidance

Abstract

We present a new interactive playback method to enhance 360° viewing experiences. Our method automatically rotates the virtual camera of a 360° panoramic video (360° video) player during interactive viewing to guide the viewer through the most important regions of the video. With this method, the viewer can watch a 360° video with minimum efforts to find important events in a scene both in interactive (e.g., HMD) and less-interactive (e.g., PC and TV) viewing environments. To estimate the importance of each viewing direction, we combine spatial and temporal saliency with cluster-based weighting. A maximum backward cumulative importance volume (MBCIV) is then constructed by accumulating this importance in the video space. During playback, which uses a forward tracing scheme through the MBCIV, the initial optimal path is found based on the viewer's viewing direction. A smooth path is then derived using penalized curve fitting. Finally, the virtual camera is rotated to follow the path. The experiments and user studies demonstrate that our method allows the viewer to effectively enjoy 360° videos with minimum interaction efforts, or even through a non-interactive display.