Light weight optical flow estimation using shuffle transformer

Abstract

Optical flow estimation is a task that estimates per pixel movement in images that comes temporally sequential. Optical flow estimation can measure the movement of objects in video, but it also works as a pretask of action recognition, frame interpolation, video super-resolution, and other various tasks. When it works as a component of other downstream tasks, there are limitations in GPU memory and inference speed. Recently, most of optical flow estimation algorithms are based on all pair cost volumes, which computes the cosine similarity of all feature pixels in two input images. This kind of algorithm can achieve surprisingly increased optical estimation performance, but it also requires a quadratic increase of memory and inference time with respect to its input resolution. In this paper, we estimate optical flow by using a multi-axis transformer and achieve comparable performance while using fewer resources such as GPU memory and inference time