(A) BCI-guided human-robot value alignment framework with action-goal inference

Abstract

e.g., one action can be associated with different goals. This makes it hard to decode human goals based on observed actions solely. Previous studies on human reinforcement learning (RL) show that humans evaluate one’s or others’ actions based on the difference between expectations and actual observations with respect to situations (state) and/or outcomes (rewards), each of which is called state prediction error (SPE) and reward prediction error (RPE). Here, we hypothesize that these two key variables in human RL, SPE and RPE, could facilitate value alignment through brain-computer interface (BCI) at both the action and goal levels. To demonstrate that SPE and RPE could be dissociated from human electroencephalography (EEG) data, we designed and conducted three independent evaluation games in which a robot navigates in the GridWorld environment. The first and second game is intended to examine the effect of SPE and RPE on actions and goals, respectively. The third game aims to evaluate their combined effect under various contexts. By analyzing the EEG data, we successfully demonstrated the significant differences in decoding accuracy between SPE and RPE across diverse frequency bands and distinct brain regions. An optimized online decoding system, built to enhance real-time computing speed by integrating the advantage of EEG-Conformer with the simple linear discriminant analysis decoder, achieved exceptional decoding speed without compromising accuracy. Furthermore, through simulations of flexible value alignment under conditions of contextual variability, we envision a promising future where the synergy between human SPE and RPE can be effectively harnessed to enhance efficacy in human-robot collaboration.

Achieving value alignment in human-robot collaboration is of great importance for improving the performance of robots through realistic engagement with humans. One critical issue is that action-goal mapping is not injective