Effects of visual auditory heart rate representation on collaboration and decision-making in a shared virtual environment

Abstract

Nonverbal cues, such as biosignal representation, are essential factors accessible in virtual reality (VR). Among various biosignals, heart rate is a salient cue enabling users to interact and communicate with each other. Though heart rate has been actively investigated in a variety of contexts, the studies were mostly conducted in 2D digital environments and in a single-player format. Taking this into consideration, the study follows a 2 by 2 within-subjects design and explores how heartbeat audio and heart rate patterns influence collaboration and decision-making in a modified Prisoner’s Dilemma game paradigm in a real-time multi-player context. The study (N = 39) shows that the heartbeat audio (on and off) and the different heart rate patterns (slow to fast and fast to slow) have an effect on collaboration (Social Presence, Trust, Cooperation Intention) and decision-making (Decision-making, Real-time Heart Rate, Choice Time) in a shared virtual environment. Results suggest that the level of Social Presence showed a significant main effect on heart rate pattern and the level of Trust showed a significant interaction effect between heart rate pattern and heartbeat audio. In addition, marginal significance was observed in terms of the interaction effect between decision-making and heart rate pattern on Choice Time. Based on this, we discuss the implications of collaboration and decision-making analyses along with responses of the in-depth interview from the participants collected after the experiment. We also provide insights on effective biosignal representation design for immersive interaction in social VR.