Development of tactile feedback system and its application to HCI and BCI

Abstract

It is a long-standing goal for many research to develop an interface system that allows humans and machines to interact in the way humans interact with each other. Nevertheless, compared to vision and audition, the sense of touch has been used very limitedly even though it is expected to improve a user's perceived veracity of the sensory experience much further. In this thesis, two types of tactile feedback systems, one for the finger pad and the other for other body parts such as torso, arms, and legs, able to output information via tactile sensation, are developed and their usages in HCI and BCI applications are explored.

In Chapter 2, a tactile feedback system for finger pad that consists of a pin-array type tactile display and a method that derives the optimal spatiotemporal patterns of a given tactile display to deliver a desired tactile perception to humans by incorporating the neurophysiological model of the finger pad are presented. The developed tactile display is composed of off-the-shelf actuators, yet has simple construction process while maintaining high stiffness and temporal bandwidth of the display. The optimization algorithm successfully found the optimal spatiotemporal stimulation patterns in both frequency and amplitude domains to present twelve inclined lines with a 5 degree difference. The cost function of the optimization algorithm was constructed as to minimize the difference between the mean firing rate of the afferents located outside of the target tactile shape and that of afferents overlapping the target shape. Human experiments revealed that the neurophysiologically optimized stimulation patterns indeed lowers the identification threshold of inclinations of the lines compared to previous studies. This result implicates that the proposed tactile stimulation system was able to deliver more detailed tactile perception with coarser tactile display by employing spatiotemporally optimized patterns.

In Chapter 3, a tactile feedback system for lower neck that consists of a 1D vibrotactile array type tactile display and a control algorithm based on tactile illusion of apparent motion and phantom sensation are presented. In addition, its application to brain-computer interfaces (BCIs) is thoroughly examined, and the possibility of extending the developed 1D tactile illusion algorithm to 2D area for HCI applications are briefly discussed. Operating a brain-actuated vehicle in a real-world environment requires much of our visual attention in observing our surroundings. However, in most of BCIs, the feedback that helps users to operate BCIs is also presented visually. As a result, users must divide their visual attention in two, which is undesirable for both operating a BCI and interacting with surroundings. Therefore, we propose a tactile feedback system that can replace visual feedback and thus free our visual channel. Six coin motors attached on the neck provided tactile feedback to users. The spatial resolution of the feedback was improved by the tactile illusion of movement. A multitasking experiment that consisted of an online BCI task and the visual multiple object tracking task was performed to investigate the advantages of tactile over visual feedback in operating a BCI. The online BCI experiment results showed that all subjects controlled the BCI with tactile feedback equally as well with visual feedback. In the multitasking experiment results, subjects tracked more targets correctly and showed higher BCI performance when tactile feedback was used. The proposed tactile feedback system enables us to use our visual channel for other purposes while operating a BCI. This could be a very important advantage when we are operating brain-actuated vehicles, especially in a complex real-world environment outside the laboratory setting.