VibAware: Context-Aware Tap and Swipe Gestures Using Bio-Acoustic Sensing

Abstract

The use of microgestures has improved the robustness and naturalness of subtle hand interactions. However, the conventional approach often neglects the context in which users perform microgestures. We present VibAware, a context-aware tap and swipe gesture recognition using bio-acoustic sensing. We use both active and passive sensing approaches to recognize finger-based microgestures while also recognizing the associated interaction contexts, including Within-Hand, Surface, and Hand Grasp. We employ accelerometers and an active acoustic transmitter to form a bio-acoustic system with multiple bandpass filter processing. Through user studies, we validate the accuracy of context-aware tap and swipe gesture recognition. We propose a context-aware microgesture recognition pipeline to enable adaptive input controls for rich and affordable hand interactions.