Konstruktor: A Strong Baseline for Simple Knowledge Graph Question Answering

Abstract

While being one of the most popular question types, simple questions such as “Who is the author of Cinderella?”, are still not completely solved. Surprisingly, even most powerful modern Large Language Models (LLMs) are prone to errors when dealing with such questions, especially when dealing with rare entities. At the same time, as an answer may be one hop away from the question entity, one can try to develop a method that uses structured knowledge graphs (KGs) to answer such questions. In this paper, we introduce Konstruktor -- an efficient and robust approach that breaks down the problem into three steps: (i) entity extraction and entity linking, (ii) relation prediction, and (iii) querying the knowledge graph. Our approach integrates language models and knowledge graphs, exploiting the power of the former and the interpretability of the latter. We experiment with two named entity recognition and entity linking methods and several relation detection techniques. We show that for relation detection, the most challenging step of the workflow, a combination of relation classification/generation and ranking outperforms other methods. On four datasets, we report the strong performance of Konstruktor.