Depth-aided disocclusion filling using exemplar-based inpainting improved by hessian matrix

Abstract

Free-viewpoint rendering (FVR) has become a popular topic in 3D research. A promising technology in FVR is to generate virtual views using a single texture image and the corresponding depth image. A critical problem that occurs when generating virtual views is that the regions covered by the foreground objects in the original view may be disoccluded in the synthesized views. In this paper, a depth based disocclusion filling algorithm using patch based texture synthesis is proposed. In contrast to the existing patch based virtual view synthesis methods, the filling priority is driven by the robust structure tensor which efficiently reflects the overall structure of an image part and a new confidence term which produces fine synthesis results even near the foreground boundaries. Moreover, the best-matched patch is searched in the background regions and finally it is chosen through a new patch distance measure. Significant superiority of the proposed method over the state-of-the-art methods is presented by comparing the experimental results. We further provide the utility of the proposed method by applying it to 3D video call system with a single camera.