Affix modification-based bilingual pivoting method for paraphrase extraction in agglutinative languages

Abstract

Paraphrase extraction is a task that involves the extraction of pairs of paraphrase expressions from a large-scale corpus. Because existing extraction methods are mostly designed for morphologically poor languages such as English, we present a method suited for agglutinative languages that are morphologically complex by attaching inflectional affixes to word stems. Specifically, we use the Korean language as a case study to address two types of problems that occur because existing methods model each lexical form as a separate word. The first problem is lexical data sparsity, and the second problem is not considering the morphological word structure. To mitigate these problems, we propose a novel phrasal paraphrase extraction method called affix modification-based bilingual pivoting method (AMBPM), which extends the existing bilingual pivoting method (BPM). Our experiments show that our proposed method significantly outperforms two state-of-the-art paraphrase extraction methods, namely the syntactic constraints-based bilingual pivoting method (SCBPM) and the skip-gram word embedding model with respect to meaning preservation and grammaticality of the extracted paraphrase pairs.