Novel edge-aware cost descriptors coupled with a fast aggregation engine for efficient multi-modal depth extraction

Abstract

Reliable and efficient stereo matching is a challenging task due to the presence of multiple radiometric variations. In stereo matching, correspondence between left and right images can become hard owing to low correlation between radiometric changes in left and right images. Previously presented cost metrics are not robust enough against intensive radiometric variations and/or are computationally expensive. In this thesis, I propose a new similarity metric coined as Intensity Guided Cost Metric (IGCM). IGCM turns out to significantly contribute to the depth accuracy by rejecting outliers and reducing the edge-fattening effect in object boundaries. IGCM is further combined explicitly with a color formation model to handle various radiometric changes that occur between stereo images. Experimental results on Middlebury dataset show 13.8%, 22.8%, 20.9%, 19.5% and 9.1% decrease in average error rate compared to Adaptive Normalized Cross-Correlation (ANCC), Dense Adaptive Self-Correlation (DASC), Adaptive Descriptor(AD), Fast Cost Volume Filtering (FCVF) and Iterative Guided Filter (IGF)-based methods, respectively. Moreover, using integral images IGCM can achieve a speedup of 20x, 6x, 41x, 25x and 45x compared to the aforementioned methods. Moreover, a novel matching cost function called REPS (Robust Edge-aware Poisson-based Similarity Metric) is proposed where poisson distribution based binary weights are utilized for a hardware-efficient cost generation for stereo matching under radiometric variations. Using REPS we can achieve comparable results to IGCM with respect to accuracy while being significantly simpler.This is validated in images with multiple radiometric variations and experimental setups. Hardware architecture utilizes an accurate log2 approximation which reduces the hardware cost significantly while maintaining the accuracy. We can achieve 33 frames per second (fps) on full-HD resolution with 128 disparity levels.