Recognizing object of attention in virtual reality

Abstract

Recent developments in Virtual Reality (VR) and Mixed Reality (MR) provide new opportunities to foster more immersive remote collaborations. In such collaboration scenarios, it is important to consider possible cues that help recognize an object of interest that users are focusing on. We propose a new data-driven approach to predict object of attention in virtual reality by making use of deictic marking information of the other user avatar as input features. Our application makes real time predictions on user attention and gaze point in virtual scenes. Our method marginally outperforms simple gaze prediction method. Further examination on object detection accuracy reveals our approach shows more robustness in dynamic scenarios for detecting object of interest validating deictic information's usefulness in gaze predictions.