Towards natural gesture interaction: design methods and case studies

Abstract

Gestures are one of the most natural interaction skills of humans. The main objective of gesture interaction is to facilitate natural and easy communication with computers by making use of this basic human skill, without any specially designed device for human-computer interaction such as the mouse or keyboard. Since gesture interactions require advanced technology and extensive computing resources are required for computers to recognize actions of users, most studies have been focused on enabling technology. Thanks to countless research on technological development and the development of computing power, gesture interaction is no longer restricted to the world of science fiction. Although gesture interaction has stepped into reality, there is still a long way to go to achieving ‘natural’. In their recent article, Malizia and Bellucci mentions that for gesture interaction to be natural, we should break down the technology-driven approach and “provide users with gestures they are more used to, taking into account their habits, backgrounds, and cultural aspects”. The question of ‘how’ to do this remains unanswered, and specific methods have yet to be introduced. In this thesis, we introduce novel design methods to derive the most natural gestures from users in the early stage of the gesture interaction design process, and to offer valuable implications for designing natural and intuitive gesture interaction by understanding users in the context of both everyday and human-computer interaction. First we introduce Interaction Probe method, which is aimed at designing gesture interaction based on manipulative gestures where the physical object is required as a medium of gestures. The method enables designers to understand a natural user-defined manipulative gesture set by establishing insights into users’ mental model according to medium characteristics. By applying Interaction Probe to design case of deformable user interfaces, we were able to gain useful design...