SyncPIE: mobile heads-up XR information access via bezel-based smartwatch touch interface

Abstract

We propose ‘SyncPIE’, an innovative mobile heads-up information system that combines AR head-worn devices with smartwatch bezel-based input to address the challenges of low accuracy and divided attention in mobile augmented reality interaction. Despite the potential of the bezel-initiated touch gesture of smartwatches for eye-free interaction in a mobile context, its application in AR environments has not yet been fully explored. The SyncPIE system includes GlancePIE, an AR interface with a three-layer hierarchical pie menu allowing users to quickly view multiple options, and TouchPIE, which includes three bezel-based smartwatch touch interfaces designed for input without looking at the watch. Among the interfaces are the bezel-initiated BTS (Bezel-to-Slide) and BTap (Bezel-Tap), along with newly developed STB (Slide-to-Bezel), all tailored for AR integration. To investigate the potential of smartwatches bezel-based mobile interaction, we conducted two studies evaluating TouchPIE’s performance, usability, and social acceptability in both standing and walking scenarios. The STB method outperformed other bezel-initiated gestures in terms of speed, accuracy and usability. In addition, when evaluating the trade-offs and benefits of dual device use, the STB method demonstrated faster and better usability as well as lower mental load and arm fatigue compared to mid-air gestures, particularly in walking scenarios. This research opens new possibilities for mobile extended reality interaction methods in smartwatch and AR environments.