Enhancing LLMs with Knowledge Graphs for Academic Literature Retrieval

Catarina Pires, Pedro Correia, Pedro Silva, Liliana Ferreira, Liliana Ferreira

2024

Abstract

While Large Language Models have demonstrated significant advancements in Natural Language Generation, they frequently produce erroneous or nonsensical texts. This phenomenon, known as hallucination, raises concerns about the reliability of Large Language Models, particularly when users seek accurate information, such as in academic literature retrieval. This paper addresses the challenge of hallucination in Large Language Models by integrating them with Knowledge Graphs using prompt engineering. We introduce GPTscholar, an initial study designed to enhance Large Language Models responses in the field of computer science academic literature retrieval. The authors manually evaluated the quality of responses and frequency of hallucinations on 40 prompts across 4 different use cases. We conclude that the approach is promising, as the system outperforms the results we obtained with gpt-3.5-turbo without Knowledge Graphs.

Paper Citation

in Harvard Style

Pires C., Correia P., Silva P. and Ferreira L. (2024). Enhancing LLMs with Knowledge Graphs for Academic Literature Retrieval. In Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR; ISBN 978-989-758-716-0, SciTePress, pages 343-347. DOI: 10.5220/0012990500003838

in Bibtex Style

@conference{kdir24,
author={Catarina Pires and Pedro Correia and Pedro Silva and Liliana Ferreira},
title={Enhancing LLMs with Knowledge Graphs for Academic Literature Retrieval},
booktitle={Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR},
year={2024},
pages={343-347},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012990500003838},
isbn={978-989-758-716-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR
TI - Enhancing LLMs with Knowledge Graphs for Academic Literature Retrieval
SN - 978-989-758-716-0
AU - Pires C.
AU - Correia P.
AU - Silva P.
AU - Ferreira L.
PY - 2024
SP - 343
EP - 347
DO - 10.5220/0012990500003838
PB - SciTePress