(The) conflict between explainability and responsibility in algorithmic decision-making

Abstract

Artificial intelligence (AI) systems are becoming pervasive in high-risk environments. Some prominent examples include the use of AI in the legal, medical, hiring, and transportation domains. These algorithms, however, are far from perfect. Algorithms deployed in the real world have shown discriminatory tendencies against racial and gender minorities, and self-driving cars have even caused the death of pedestrians. A natural question that arises is who should be held responsible when AI systems cause harm to individuals. Scholars contend that the technological and organizational complexity involved in developing and deploying AI in high-risk scenarios makes determining who is responsible difficult, if not impossible, creating a ''responsibility gap.'' One of the sources of this gap is the opacity of current AI systems. Users, policymakers, and even developers do not understand how algorithms make decisions, making it difficult to identify the root cause of the harm. This limitation has fueled research on Explainable AI (XAI), which attempts to develop systems that provide explanations for their decisions and actions, helping to establish who is responsible for AI-caused harm. In contrast to the prevalent perspective that XAI could help bridge the responsibility gap in the context of algorithmic decision-making, I argue that algorithms providing post-hoc explanations could complicate the search for a responsible entity. I first show how post-hoc explanations could create the perception that AI systems (i.e., decision-making algorithms) and those subjected to algorithmic decisions (i.e., patients) are responsible when things go wrong. This perception can be exploited by AI developers, who can use these mistaken intuitions to escape deserved responsibility by using algorithms and patients as scapegoats. I then present three empirical studies (N = 1,153) exploring whether providing simple post-hoc explanations impacts the extent to which laypeople blame AI systems, their users, and developers for AI-caused harm. The findings suggest that while explainability alone does not affect folk perceptions of responsibility, specific types of explanations influence whom laypeople blame for AI-caused harm. I also explore what determines laypeople's blame judgments of AI and show that moral judgments of algorithms have unique features that are not found in judgments of human actors. Finally, I discuss how the control that developers have over explainable algorithms may allow them to implement specific post-hoc explanations that shift perceived responsibility to other actors, potentially impacting how AI is regulated. I show how this conflict between explainability and responsibility may be dealt with during the development of AI systems and defend hard regulation to prevent scapegoating.