Comparing pointer, sketch and hand gesture in the 3D workspace with MR remote collaboration system

Abstract

Remote collaboration system assists remote helpers in providing professional instructions to local workers for real-world tasks. Many researchers have studied the effect of visual cues (e.g. Pointer, Sketch and Gesture) with fixed 2D screen for real-world tasks, however, the different perspectives of these cues in the 3D workspace among two sides have not been well investigated. In this thesis, we implemented a prototype Mixed Reality (MR) remote collaboration system to help participants complete Target Location Placement task, and conducted a user study to collect data in terms of completion time, usability, feeling of co-presence and workload. The study results revealed that both sides prefer the Gesture since compared with Pointer it significantly improves the completion time as well as usability and relieves workload. Sketch with a permanent trajectory helps users review the previous instructions. With different manipulation purposes, local workers have a higher feeling of co-presence and lower workload than remote helpers, while there is no deviation in usability between both sides. On the base of these findings, we made the design suggestions for the future MR remote collaboration system and discuss the following work direction.