Triple-Frame-Based Bi-Directional Motion Estimation for Motion-Compensated Frame Interpolation

Abstract

In this paper, new motion estimation (ME) and motion vector refinement (MVR) methods for motion-compensated frame interpolation (MCFI) are proposed. To estimate motion vectors (MVs) with great reliability, triple-frame-based bi-directional ME (TFBME), which employs three sequential frames, is developed. In TFBME, the middle frame works as a reference to estimate MVs between two end frames, which leads to performance improvement especially when objects move linearly through three sequential frames. In addition, a modified bi-directional ME method, which employs only two successive frames, is combined with TFBME to handle cases in which objects move non-linearly through three sequential frames. In our proposed MVR, artifacts on interpolated frames are first detected by a convolutional neural network and then corrected by weighted vector median filtering. Experimental results show that the proposed MCFI method outperforms conventional methods in both subjective and objective evaluations.