Video frame interpolation using complex motion pattern aware deep learning

Abstract

In the field of computer vision and video processing, video frame interpolation has received significant interest. Motion of objects in video could be smooth and continuous by synthesizing intermediate frame. When the motion is complex, large and non-linear, generated intermediate frame is blurred and has large artifacts. In this work, we propose a novel video frame interpolation considering the complex motion patterns. The proposed video frame interpolation takes into account an complex motion map that contains location and intensity of the complex motion. The proposed method consists of three parts, which are optical flow based frame interpolation, complex motion detection and frame refinement. The proposed video frame interpolation can be robust against the complex motion including irregular and large motion. Experimental results showed that the proposed video frame interpolation achieved high performance on various public video dataset and especially on videos with complex motion patterns.