Knowledge-Augmented Large Language Model Prompting for Cypher Query Construction

Abstract

While existing large language models demonstrate remarkable proficiency inn a turall a nguage processing tasks, their capacity for knowledge graph structuring remains a potential avenue for enhancement, particularly in the context of knowledge graph question answering. This paper presents a novel method for knowledge-enriched LLM routing for the construction of Cypher queries utilising knowledge graphs. It is proposed that the prompts of LLMs be enriched with knowledge by the addition of relevant triples taken from the knowledge graph. The proposed method facilitates the generation of more precise and pertinent Cypher queries by LLMs, thereby enhancing their question-answering capabilities. The method is evaluated using a story-based knowledge graph question answering dataset, which is introduced in this paper as a new contribution to the literature. The results demonstrate that the incorporation of knowledge enhances the performance of knowledge graph question answering (KGQA), particularly in the context of complex and temporal inquiries. Furthermore, the utilisation of a story graph structure for the modelling of events in news texts facilitates the effective resolution of complex questions by following the paths on the graph. Finally, it is demonstrated that the extraction of temporal labels and their incorporation into the knowledge graph is of paramount importance in answering temporal questions. © 2025 Elsevier B.V., All rights reserved.