Using LLMs to Extract Adverse Drug Reaction (ADR) from Short Text

Monika Gope, John Wang

2025

Abstract

Adverse drug reactions (ADRs) are unexpected negative effects of a medication despite being used at its normal dose. Awareness of ADRs can help pharmaceutical companies refine drug formulations or adjust dosing guidelines to make medications safer and more effective. Twitter (X) can be a handy platform to extract unbiased ADR data from a large and diverse group of people. However, extracting ADRs from short texts such as tweets presents challenges due to the informal, noisy, and diverse nature of the text, which includes variations in user language, abbreviations, and misspellings. These factors make it difficult to accurately identify ADRs. Hence, it is important to identify the most effective strategies for extracting reliable ADR information. In this paper, we comprehensively evaluate various large language models (LLMs) and ML approaches for ADR extraction and detection. Using multiple ADR datasets and a range of prompt formulations, we compare the performance of each model. By systematically testing the effectiveness of these techniques across different combinations of models, datasets, and prompts, we aim to identify the most effective strategies for extracting reliable ADR information. Our study shows that LLMs excel in extracting ADRs, for example, with GPT-4 achieving an F1 score of 0.82, surpassing the previous ML methods of 0.64 for the SMM4H dataset. This indicates that LLMs are more effective and simpler alternatives to machine learning models for ADR extraction.

Paper Citation

in Harvard Style

Gope M. and Wang J. (2025). Using LLMs to Extract Adverse Drug Reaction (ADR) from Short Text. In Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF; ISBN 978-989-758-731-3, SciTePress, pages 548-555. DOI: 10.5220/0013160700003911

in Bibtex Style

@conference{healthinf25,
author={Monika Gope and John Wang},
title={Using LLMs to Extract Adverse Drug Reaction (ADR) from Short Text},
booktitle={Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF},
year={2025},
pages={548-555},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013160700003911},
isbn={978-989-758-731-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF
TI - Using LLMs to Extract Adverse Drug Reaction (ADR) from Short Text
SN - 978-989-758-731-3
AU - Gope M.
AU - Wang J.
PY - 2025
SP - 548
EP - 555
DO - 10.5220/0013160700003911
PB - SciTePress