Data-driven image expansion using structure aware image warping

Abstract

We significantly expand the field of view of a photograph by structural correlation analysis of photos taken from completely different places on earth. Our method can extrapolate typical photos into complete panoramas by pasting the original image to structurally similar images. The extrapolation problem is formulated in the structure-driven image warping framework. We evaluate likely correspondences between salient structures of both an input image and a searched image with an initial HOG position to generate a transformation for aligning them more precisely. A structure guided image warping method is then proposed to make a composite of two completely different images while preserving the input image. Different from conventional composition methods, our method builds upon a guide curve synthesis process in order to smoothly bridge two different structures while inheriting their characteristics through our novel curve shape transfer method. The guide curve is utilized to warp the searched image and to combine two different images seamlessly. We demonstrate our approach on a variety of natural scenes and compare our results with other extrapolation algorithms.