Understanding how a conversational agent should respond to users' verbal abuse: a multimodal approach to designing emotionally intelligent agent

Abstract

With the popularity of AI-infused systems, conversational agents (CAs) are becoming essential in diverse areas, offering new functionality and convenience but simultaneously affected by misuse as well as verbal abuse. In this study, we investigate how conversational agents can effectively respond to a user’s verbal abuse through a series of studies. More specifically, we performed four studies involving three different modalities (text, voice, and animated agent with voice). In Study 1, we examined whether a CA’s response style (avoidance, empathy, and counterattacking) and abuse type (insult, threat, and swear) affect the emotions (e.g., guilt and shame) that mitigate people’s aggressive behavior as well as users’ evaluations of the agents (e.g., likability and intelligence) withinthecontextoftext-basedagents. InStudy2,weprimarilyreplicatedapreviousstudybutwithinthecontext of voice-based agents. In addition, we assessed extant Intelligent Personal Assistant (IPA) responses to verbal abuse to identify the current status of the issue investigated in this study. A common finding from the two studies is that a CA’s response style, in particular empathy, is a significant factor affecting a user’s moral emotions and agentevaluations. Participantsfeltlessangryandguiltierwiththeempatheticagentthanwiththeothertwoagents. Users assessed the responses from the counterattacking agent or avoidance agent as less appropriate. This finding was consistent across text- and voice-based interaction environments. Subsequently, in Study 3, we investigated empathy and compared the effects of two alternative types of empathetic agents (other-oriented vs. self-oriented) on the emotions and agent evaluations analyzed before, in addition to the facial expressivity of the agent as a new potential influencing factor. In Study 4, we validated the effectiveness of key results obtained in a laboratory environment through a user prototype bot field test performed in a real environment. Our study has found that major CAs are generally adopting the strategy of ignoring or avoid its user’s verbal abuses. Our research findings indicate that the approaches employed by the current major CAs were ineffective in handling users’ verbal abuse. The agent’s empathetic response may be particularly useful for customer service agents, especially in managing consumers’ negative emotions. In general, our findings facilitate the design of CAs and highlight the need for appropriate strategies to address the abusive utterances of users.