Deep learning-based solutions for empowering visual localization and other vision tasks

Abstract

Visual localization is essential for many applications, including AR/VR, robots, and self-driving cars. Traditional methods use large memory and processing resources to estimate the camera position in absolute and relative terms. It gave rise to a new pattern of finding the pose using learning-based methods, i.e., pose regressors. Existing relative camera pose estimation techniques rely solely on balancing hyperparameter tuning manually or automatically in the loss function. On the other hand, current absolute pose regressors generally lack the quality to adapt to different domains of the same scene. In this work, we primarily address these two issues. First, estimating the relative camera position between a pair of images is formulated using a two-stage training strategy that eliminates the need for compensating hyperparameters in the loss function. Our proposed training strategy drastically improved the translation vector estimation by 16.11%, 28.88%, and 52.27% on the KingsCollege, OldHospital, and StMarysChurch scenes, respectively. To demonstrate texture invariance, we explore the generalization of the proposed method by extending the datasets to different scene styles for ablation and qualitative studies using Generative Adversarial Networks(GAN). Second, we offer a novel lightweight domain adaptive training framework to retrain any existing absolute pose regressors(APR) to improve their generalization capability. Our lightweight network outperforms the transformer in translation vector estimation on the visual localization benchmark dataset. The results show that despite using about 24 times fewer FLOPs, 12 times fewer activations, and five times fewer parameters than state-of-the-art MS-Transformer, our approach outperforms all CNN-based architectures and achieves comparable performance to transformer-based architectures. Our method achieves ranks 2nd and 4th with the Cambridge Landmarks and 7Scenes datasets, respectively. Moreover, our approach outperforms and ranks 1st over the MS -transformer on unseen domains. Furthermore, This work explores the demonstration of an APR's inversion for synthesizing views similar to NeRF.