FISR: Deep Joint Frame Interpolation and Super-Resolution with a Multi-Scale Temporal Loss

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 28
  • Download : 0
Super-resolution (SR) has been widely used to convert lowresolution legacy videos to high-resolution (HR) ones, to suit the increasing resolution of displays (e.g. UHD TVs). However, it becomes easier for humans to notice motion artifacts (e.g. motion judder) in HR videos being rendered on largersized display devices. Thus, broadcasting standards support higher frame rates for UHD (Ultra High Definition) videos (4K@60 fps, 8K@120 fps), meaning that applying SR only is insufficient to produce genuine high quality videos. Hence, to up-convert legacy videos for realistic applications, not only SR but also video frame interpolation (VFI) is necessitated. In this paper, we first propose a joint VFI-SR framework for upscaling the spatio-temporal resolution of videos from 2K 30fps to 4K 60 fps. For this, we propose a novel training scheme with a multi-scale temporal loss that imposes temporal regularization on the input video sequence, which can be applied to any general video-related task. The proposed structure is analyzed in depth with extensive experiments.
Publisher
The Association for the Advancement of Artificial Intelligence
Issue Date
2020-02-11
Language
English
Citation

Thirty-Fourth AAAI Conference on Artificial Intelligence, pp.11278 - 11286

ISSN
2374-3468
DOI
10.1609/aaai.v34i07.6788
URI
http://hdl.handle.net/10203/278754
Appears in Collection
EE-Conference Papers(학술회의논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0