Enriching Mobile Touch Interaction using Normal and Tangential Forces

Abstract

Advances in technology brought powerful and connected computers to our pockets, and the touchscreen has been the main interface for these computers. The touch input, however, is limited in its expressivity, only allowing a 2D position of a finger contact as input. As mobile devices are handling more tasks than ever before, the limited expressivity of the touch input requires users to use on-screen buttons to switch between touch modes. In fact, our natural finger touch is rich and continuous. A finger touch can express different intentions and nuances by controlling normal and tangential forces, and we also feel the response of an object through the touch. This thesis describes our exploration of using normal and tangential forces to enrich touch interaction. I present 1) ForceTap, a method to enrich tap gesture with a normal force, 2) ForceDrag, an interaction technique to use a normal force as a drag gesture modifier, 3) Force Gestures, a set of touch gestures enriched with normal and tangential forces, and 4) a method to detect and use multi-point tangential force input. Evaluations showed that the normal and tangential forces could successfully enrich touch interaction, however, also found that the lack of physical movement during the force interaction creates frustration and high physical demand. We developed a haptic feedback technique that uses vibration to create an illusion of compliant feeling on a rigid surface. Experiments showed that the illusion was perceived as a real movement and reduced task load of multi-dimensional force interaction.