Deep joint deblurring and multi-frame interpolation with flow-guided attentive correlation and recursive boosting흐름 유도 기반의 주의 상관 관계 및 재귀 부스팅을 이용한 심층 합동 디블러링 및 다중 프레임 보간
Video frame interpolation (VFI) synthesizes intermediate frames to temporally upscale a low frame rate (LFR)
video to a high frame rate (HFR) one, which provides a visually pleasing experiences to users. On the other
hand, motion blur is generally induced when capturing videos due to the accumulations of the light. Therefore,
eliminating the motion blur, called deblurring, is also important to synthesize sharp intermediate frames while
performing VFI. In this dissertation, we propose a novel joint deblurring and multi-frame interpolation (DeMFI)
framework, called DeMFI-Net, which accurately converts blurry videos of lower-frame-rate to sharp videos at
higher-frame-rate based on flow-guided attentive-correlation-based feature bolstering (FAC-FB) module and recursive boosting (RB), in terms of multi-frame interpolation (MFI). The DeMFI-Net jointly performs deblurring
and MFI where its baseline version performs feature-flow-based warping with FAC-FB module to obtain a sharpinterpolated frame as well to deblur two center-input frames. Moreover, its extended version further improves the
joint task performance based on pixel-flow-based warping with GRU-based RB. Our FAC-FB module effectively
gathers the distributed blurry pixel information over blurry input frames in feature-domain to improve the overall
joint performances, which is computationally efficient since its attentive correlation is only focused pointwise.
Furthermore, we conduct diverse experiments on module locations, occlusion maps, efficient version, one-stage
version and adaptivity on fps to deeply study our network for DeMFI. As a result, our DeMFI-Net achieves
state-of-the-art (SOTA) performances for diverse datasets with significant margins compared to the recent SOTA
methods, for both deblurring and MFI. All source codes including pretrained DeMFI-Net are publicly available at
https://github.com/JihyongOh/DeMFI.