Deep learning based weighted multi-kernel prediction network for burst image super-resolution

Abstract

Burst image super-resolution is an ill-posed problem that aims to restore a high-resolution (HR) image from a sequence of low-resolution (LR) burst images. To restore a photo-realistic HR image using their abundant information, it is essential to align each burst of frames containing random hand-held motion. Some kernel prediction networks (KPNs) that are operated without external motion compensation such as optical flow estimation have been applied to burst image processing as implicit image alignment modules. However, the existing methods do not consider the interdependencies among the kernels of different sizes that have a significant effect on each pixel. In this paper, we propose a novel weighted multi-kernel prediction network (WMKPN) that can learn the discriminative features on each pixel for burst image super-resolution. Our experimental results demonstrate that WMKPN improves the visual quality of super-resolved images. To the best of our knowledge, it outperforms the state-of-the-art within kernel prediction methods and multiple frame super-resolution (MFSR) on both the Zurich RAW to RGB and BurstSR datasets.