Comparing Large Language Models for Automated Subject Line Generation in e-Mental Health: A Performance Study

Philipp Steigerwald, Jens Albrecht

2025

Abstract

Large Language Models (LLMs) have the potential to enhance e-mental health and psychosocial e-mail coun-selling by automating tasks such as generating concise and relevant subject lines for client communications. However, concerns regarding accuracy, reliability, data privacy and resource efficiency persist. This study investigates the performance of several LLMs in generating subject lines for e-mail threads, yielding a total of 253 generated subjects. Each subject line was assessed by six raters, including five counselling professionals and one AI system, using a three-category quality scale (Good, Fair, Poor). The results show that LLMs can generally produce concise subject lines considered helpful by experts. While GPT-4o and GPT-3.5 Turbo outperformed other models, their use is restricted in mental health settings due to data protection concerns, making the evaluation of open-source models crucial. Among open-source models, SauerkrautLM LLama 3 70b (4-bit) and SauerkrautLM Mixtral 8x7b (both 8-bit and 4-bit versions) delivered promising results with potential for further development. In contrast, models with lower parameter counts produced predominantly poor outputs.

Paper Citation

in Harvard Style

Steigerwald P. and Albrecht J. (2025). Comparing Large Language Models for Automated Subject Line Generation in e-Mental Health: A Performance Study. In Proceedings of the 11th International Conference on Information and Communication Technologies for Ageing Well and e-Health - Volume 1: ICT4AWE; ISBN 978-989-758-743-6, SciTePress, pages 70-77. DOI: 10.5220/0013294100003938

in Bibtex Style

@conference{ict4awe25,
author={Philipp Steigerwald and Jens Albrecht},
title={Comparing Large Language Models for Automated Subject Line Generation in e-Mental Health: A Performance Study},
booktitle={Proceedings of the 11th International Conference on Information and Communication Technologies for Ageing Well and e-Health - Volume 1: ICT4AWE},
year={2025},
pages={70-77},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013294100003938},
isbn={978-989-758-743-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 11th International Conference on Information and Communication Technologies for Ageing Well and e-Health - Volume 1: ICT4AWE
TI - Comparing Large Language Models for Automated Subject Line Generation in e-Mental Health: A Performance Study
SN - 978-989-758-743-6
AU - Steigerwald P.
AU - Albrecht J.
PY - 2025
SP - 70
EP - 77
DO - 10.5220/0013294100003938
PB - SciTePress