Stable and fast novel view synthesis with few shot images

Abstract

Novel View Synthesis is a long-standing problem for computer vision and Robotics applications. Neural Radiance Fields (NeRF) recently introduced a method that can synthesize novel views by optimizing volumetric scene function with given images. However, NeRF degenerates when optimized with few input views because the scene function tends to be overfitted to a few images regardless of geometric constraints. Moreover, it takes a long time to build a volumetric function to render proper novel view radiance. To handle these issues, we propose fast few-shot NeRF on recently advanced voxel grids to synthesize novel views with geometric cues. We utilize two strong geometric cues which can be captured from monocular RGB by using recent advances in deep dense priors estimation, the depth map, and surface normal. In addition, we utilized multi-view consistency to solve the up-to-scale problem of monocular depth prediction. The naive approach to optimize jointly surface normal with neural implicit representation is a differentiable Signed Distance Function (SDF) with eikonal loss. However, we found that eikonal loss does not help to optimize with few views in complex geometry, so we adapt SDF loss which can make geometry smooth. This simple approach allows plausible performance with more than 30 times faster optimization time than state-of-the-art few-shot novel view synthesis methods.