Hand occlusion-free palm force feedback rendering with wrist-worn device

Abstract

In virtual reality (VR), users conduct various interactions using their hands. During hand interaction, users expect haptic feedback, and one of the easiest ways to deliver realistic haptic feedback is by wearing a haptic device. However, there is an issue where the device blocks the camera’s tracking, resulting in occlusion. Such tracking loss makes seamless force feedback rendering difficult, degrading the user experience. Therefore, in this thesis, we introduce a method to predict wrist position by integrating an IMU-based tracking approach into a wrist-worn haptic device to render palm force feedback. Additionally, we developed a rendering algorithm that enables a seamless transition between IMU-based tracking and camera-based tracking. The IMU-based tracking method was designed using a Long Short-Term memory model (LSTM), and the haptic rendering algorithm is designed considering user experience. We validated our proposed method's usage systematically in various scenarios.