Analyzing stereotypical persona bias in personalized dialogue generation models

Abstract

Dialogue agents that reenact existing gender stereotypes can cause harm by propagating those stereotypes. This study shows that providing a language model fine-tuned for personalized dialogue generation with a certain gender as its explicit persona biases its implicit persona in a direction that conforms to traditional gender stereotypes. To show this we collect a set of human generated questions that asks about the characteristics of a stereotypical male or female. Then, with a binary classifier that can distinguish between stereotypical and non-stereotypical answers, we compare the percentage of stereotypical answers generated by dialogue models given different personas. We find that personalized dialogue generation models are more likely to generate answers that conform to a certain gender's stereotypes when given that gender as their persona, compared to answers generated either without an explicit persona or with the opposite gender as its explicit persona.