Understanding user internal states for authentication and human-computer interface

Abstract

Understanding human internal states is an important factor of successful human-computer interactions. In this dissertation, we suggest to analyze various bio-signals to understand human internal states even when they are not explicitly presented. Among various user states we focused on preferences for authentication and trust during cooperation with human-like machines.

First, we measured and analyzed idiosyncratic eye movements elicited from attention shifts in aspect of security codes which is fundamentally impossible for intruders to reproduce as genuine owners. In our experiment, sequences of visual stimuli were presented, and corresponding eye movements were recorded. We hypothesized that these eye movements evoked from the stimuli, and that each signal sequence can be composed into code that is unique to each person. To verify this hypothesis, we conducted eye movements comparisons between subjects and authentication of subjects with eye movements. Obtained results imply that eye movements on each visual stimulus can be considered as idiosyncratic, and compositions of multiple responses can provide more reliable information for subject authentication. Efficiencies in authentication could be improved with selectively presenting stimuli which induced more consistent scanpaths from a claimed identity, and the best performance was achieved at a 1.57% false acceptance and a 1.23% false rejection rate. This supports the feasibilities of proposed security system based on eye scanning paths on visual stimuli.

The second part of this dissertation is understanding human trust in machines and exploring factors of the trust by observing human brain activities. Recently, automated systems (self-driving cars, autopilot, etc.) operated under human supervisions have become increasingly ubiquitous. However, little is known about the underlying neural and computational processes of how human operator supervises partner agents' decisions. We conducted a novel experiment to model human trust in machines with various human-like cues. In our experiment, each subject performed a task as a player or coach for machine players. We could demonstrate the significant differences in event related potentials according to subjects’ trust in the agents. Also, we could observe neural correlates to variations of human trust in trial-level by using statistical analysis and classification with a machine-learning method. Variations in brain activities related to trust changes were pronounced for externally more human-like agents. Moreover, subjects tended to trust more in agents with similar risk-taking personalities to themselves. This research provides a theoretical basis for modelling human neural activities indicate trust in partner machines and can thereby contribute to the design of machines to promote efficient interactions with humans.