Augmented SLAM and object detection framework for visually impaired people assistance

Abstract

In this thesis we tackle the problem of assisting the visually impaired using the latest methods in Simultaneous Localization and Mapping (SLAM) and a deep learning-based object detection. We implement information from a map constructed using ORB-SLAM2 to a newly generated map which contains detected objects. Using the map and detected object combination we let the user know the whereabouts of the object of interest and also construct a detailed relative scene graph. We provide extensive experiments to prove our concept. The results show the we are able to achieve high degrees of localization accuracy for locating objects in a sparse map provided by out ORB-SLAM2 system.