Robust and efficient image alignment through simultaneous low rank and sparse decomposition

Abstract

Accurate alignment of a set of images is often hindered by various forms of image corruptions such as noise and occlusion. To address the problem, we propose an algorithm named REALS for robust and efficient batch image alignment through simultaneous low rank and sparse decomposition. It is based on two operations, geometric transformation and low rank and sparse decomposition, combined in a form that we can backpropagate through them, which enables simultaneous image alignment and decomposition with gradient-based updates. We show that REALS achieves an order of magnitude improvement in terms of accuracy and speed compared to the state-of-the-art methods. In addition, we demonstrate its capability by aligning neural activity imaging datasets with a high level of motion artifacts, noise and neural activities.