A Unique Training Strategy to Enhance Language Models Capabilities for Health Mention Detection from Social Media Content

Pervaiz Khan, Pervaiz Khan, Muhammad Asim, Andreas Dengel, Andreas Dengel, Sheraz Ahmed

2024

Abstract

An ever-increasing amount of social media content requires advanced AI-based computer programs capable of extracting useful information. Specifically, the extraction of health-related content from social media is useful for the development of diverse types of applications including disease spread, mortality rate prediction, and finding the impact of diverse types of drugs on diverse types of diseases. Language models are competent in extracting the syntactic and semantics of text. However, they face a hard time extracting similar patterns from social media texts. The primary reason for this shortfall lies in the non-standardized writing style commonly employed by social media users. Following the need for an optimal language model competent in extracting useful patterns from social media text, the key goal of this paper is to train language models in such a way that they learn to derive generalized patterns. The key goal is achieved through the incorporation of random weighted perturbation and contrastive learning strategies. On top of a unique training strategy, a meta predictor is proposed that reaps the benefits of 5 different language models for discriminating posts of social media text into non-health and health-related classes. Comprehensive experimentation across 3 public benchmark datasets reveals that the proposed training strategy improves the performance of the language models up to 3.87%, in terms of F1-score, as compared to their performance with traditional training. Furthermore, the proposed meta predictor outperforms existing health mention classification predictors across all 3 benchmark datasets.

Paper Citation

in Harvard Style

Khan P., Asim M., Dengel A. and Ahmed S. (2024). A Unique Training Strategy to Enhance Language Models Capabilities for Health Mention Detection from Social Media Content. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-680-4, SciTePress, pages 673-680. DOI: 10.5220/0012384100003636

in Bibtex Style

@conference{icaart24,
author={Pervaiz Khan and Muhammad Asim and Andreas Dengel and Sheraz Ahmed},
title={A Unique Training Strategy to Enhance Language Models Capabilities for Health Mention Detection from Social Media Content},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2024},
pages={673-680},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012384100003636},
isbn={978-989-758-680-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - A Unique Training Strategy to Enhance Language Models Capabilities for Health Mention Detection from Social Media Content
SN - 978-989-758-680-4
AU - Khan P.
AU - Asim M.
AU - Dengel A.
AU - Ahmed S.
PY - 2024
SP - 673
EP - 680
DO - 10.5220/0012384100003636
PB - SciTePress