In this paper, we propose a concept for depth estimation for constructing realistic panorama system using sparse image set. First, we find
overlapping regions between adjacent input images. Second, we estimate depth information of overlapping regions using stereo information and relative height depth cue. Finally, we propagate depth information from overlapping regions to non-overlapping regions using relative height depth cue.