Implicit emotional processing and its individual multimodal classification using event-related potential and skin response

Abstract

In daily life, emotional processing usually occurs unconsciously and automatically and it has an influence on ongoing attentional processing. Although emotional processing plays a role of a facilitator or distractor for main attentional processing, direction of its influence is complex and controversial. In this study, specific types of emotional stimuli were selected via galvanic skin response (GSR) and event-related potential (ERP) during implicit emotional processing. Moreover, we tried to find how implicit emotional processing will have effects on attentional processing in terms of individual differences. To induce implicit emotional processing, the emotional picture was presented briefly (80ms) immediately before every trial of an attention task where reaction time (RT) was measured. There were five kinds of emotional conditions (neutral, positive, erotic, mutilation, and horror), and EEG and GSR were recorded during the whole task period. We found increased GSR in the horror condition and increased P300 and late positive potential (LPP) amplitude in the erotic and horror conditions. Further, participants were divided into two groups depending on whether RT in these conditions was faster or slower than that in the neutral condition. Divided participants who did not show emotional effects on ERP responses in each emotional condition showed faster RT in the erotic condition and slower RT in the horror condition. However, participants whose P300 amplitude increased showed slower RT in the erotic condition, but participants whose both P300 and LPP amplitudes increased showed faster RT in the horror condition. Finally, in these emotional conditions, it was possible to predict emotional effect on RT via machine learning using multimodal features including GSR and ERP. According to previous studies, it seems that pleasant emotion has positive effects and unpleasant emotion has negative effects on attentional processing. In this study, furthermore, we found that emotional effects on attentional processing could be opposite in the pleasant and unpleasant condition when participants significantly captured emotional significance (P300) or regulated their emotion (LPP).