A BERT-enhanced Graph Neural Network for Knowledge Base Population

Abstract

We present BGKBP, a deep-learning algorithm based on BERT, and a graph neural network for knowledge base population (KBP). Our experiments showed that a straightforward application of BERT and GNN on a large knowledge base (e.g., Wikidata) improves KBP quality and outperforms the previous state-of-the-art methods. We developed four techniques to improve the BGKBP’s KBP capability: (1) serialization, (2) fine-tuning, (3) node regression, and (4) text augmentation. BGKBP achieved the best F1 scores of 0.723 and 0.495 on entity linking and new entity detection, respectively. We further showed that using text augmentation (BGKBP-TA) achieved the best F1 score of 0.547 on relation linking, which is more difficult than entity linking because of the various representations of some of the relations.