Addressing factual error in abstractive dialogue summarization via span identification and correction

Abstract

Abstractive dialogue summarization presents unique challenges due to the dynamic nature of conversations, involving multiple speakers, role changes, language variations, and informalities. Despite recent advancements in this field, summaries generated by existing methods often suffer from factual errors. To address this issue, post-processing correction has emerged as a promising approach that offers practicality and can be combined with other techniques. However, existing correction models still exhibit limitations, including false corrections that transform clean summaries into incorrect ones. We propose "identify-then-correct" framework as a novel foundation for post-processing correction in abstractive dialogue summarization. Our framework comprises three steps: decide, identify, and correct. Initially, the framework determines whether a summary contains factual errors and proceeds to identify the wrong part. This identified segment then serves as guidance for the correction. Our evaluation results demonstrated the effectiveness of our identifier and corrector model in terms of detecting incorrect summaries and correction, proof that providing guidance from the identifier model improved correction performance. The overall evaluation showed that our framework outperformed the baseline in balancing identification and correction while also highlighting its flexibility and practicality as it allows for the integration of models that align with its objectives. Furthermore, the factuality human evaluation provided further validation of the effectiveness of our models and framework, emphasizing the ability of our approach to achieve accurate correction while preventing false correction.