Authors:
Awais Khan Nawabi
1
;
Janos Tolgyesi
2
;
Elena Bianchi
3
;
Chiara Toffanin
1
and
Piercarlo Dondi
1
Affiliations:
1
Department of Electrical, Computer, and Biomedical Engineering, University of Pavia, Via Ferrata 5, 27100, Pavia, Italy
;
2
Neosperience, Via Privata Decemviri 20, 20137, Milano, Italy
;
3
Neosperience Health, Via Privata Decemviri 20, 20137, Milano, Italy
Keyword(s):
Large Language Model, Retrieval-Augmented Generation, Prompt Engineering, User Study, Diabetes.
Abstract:
Diabetes is a common chronic illness projected to increase significantly in the coming years. Managing diabetes is complex, requiring patients to frequently adjust their treatments and lifestyles to prevent complications. Awareness and adherence to healthy habits are thus essential. Artificial Intelligence (AI) can assist in this effort. Recent advancements in Large Language Models (LLMs) have enabled the creation of effective chatbots to support patients. However, despite their growing use, there are still a few formal user studies on LLMs for diabetes patients. This study aims to investigate the ability of an LLM-based chatbot to provide useful and understandable information to potential patients. Specifically, the goal was to examine how variations in language and wording affect the comprehension and perceived usability of the chatbot. To this end, D-Care, a chatbot assistant based on OpenAI’s ChatGPT-4o, was developed. D-Care can generate answers in four different tones of voice,
ranging from elementary to technical language. A user study with 40 participants showed that changes in tone can indeed impact the system’s comprehension and usability.
(More)