Leveraging Cross-Verification to Enhance Zero-Shot Prompting for Care Document Data Extraction

Laura Steffny, Nanna Dahlem, Robert Becker, Dirk Werth

2025

Abstract

Automating care documentation through artificial intelligence (AI), particularly using large language models (LLMs), has great potential to improve workflow and efficiency in healthcare applications. However, in clinical or care environments where errors can have serious consequences, ensuring the reliability and accuracy of LLM output is essential. Zero-shot prompting, an advanced technique that does not require task-specific training data, shows promising results for data extraction in domains where large, well-structured datasets are scarce. This paper investigates how cross-verification affects zero-shot prompting performance in extracting relevant care indicators from unbalanced nursing documentation. The extraction was evaluated for three indicators on a dataset of care documentation from 38 participants across two facilities. The results show cross-verification significantly improves extraction accuracy, particularly by reducing false positives. While term extraction alone achieved around 80% accuracy, at lower temperature settings (0.1) cross-verification increased accuracy to 96.74%. However, cross-verification also increased missed terms when no corresponding sentences were found, even though terms were in the ground truth. This study highlights the potential of cross-verification in care documentation and offers suggestions for further optimization, especially with unstructured text and unbalanced data.

Paper Citation

in Harvard Style

Steffny L., Dahlem N., Becker R. and Werth D. (2025). Leveraging Cross-Verification to Enhance Zero-Shot Prompting for Care Document Data Extraction. In Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF; ISBN 978-989-758-731-3, SciTePress, pages 115-128. DOI: 10.5220/0013177600003911

in Bibtex Style

@conference{healthinf25,
author={Laura Steffny and Nanna Dahlem and Robert Becker and Dirk Werth},
title={Leveraging Cross-Verification to Enhance Zero-Shot Prompting for Care Document Data Extraction},
booktitle={Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF},
year={2025},
pages={115-128},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013177600003911},
isbn={978-989-758-731-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF
TI - Leveraging Cross-Verification to Enhance Zero-Shot Prompting for Care Document Data Extraction
SN - 978-989-758-731-3
AU - Steffny L.
AU - Dahlem N.
AU - Becker R.
AU - Werth D.
PY - 2025
SP - 115
EP - 128
DO - 10.5220/0013177600003911
PB - SciTePress