Comparative Analysis of the Efficacy in the Classification of Cognitive Distortions Using LLMs

Aaron Pico, Joaquin Taverner, Emilio Vivancos, Ana Garcia-Fornes

2025

Abstract

This paper explores the application of Large Language Models (LLMs) for the classification of cognitive distortions in humans. This is important for detecting irrational thought patterns that may negatively influence people’s emotional state. To achieve this, we evaluated a range of open-source LLMs with varying sizes and architectures to assess their effectiveness in the task. The results show promising results of the recognition capabilities of these models, particularly given that none of them were specifically fine-tuned for this task and were solely guided by a structured prompt. The results allow us to see a trend where larger models generally outperform their smaller counterparts in this task. However, architecture and training strategies are also important factors, as some smaller models achieve performance levels comparable to or exceeding larger ones. This study has also allowed us to see the limitations in this field: the subjectivity factor that may exist in the annotations of cognitive distortions due to overlapping categories. This ambiguity impacts both human agreement and model performance. Therefore, future work includes fine-tuning LLMs specifically for this task and improving the quality of the dataset to improve performance and address ambiguity.

Paper Citation

in Harvard Style

Pico A., Taverner J., Vivancos E. and Garcia-Fornes A. (2025). Comparative Analysis of the Efficacy in the Classification of Cognitive Distortions Using LLMs. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: EAA; ISBN 978-989-758-737-5, SciTePress, pages 957-965. DOI: 10.5220/0013399200003890

in Bibtex Style

@conference{eaa25,
author={Aaron Pico and Joaquin Taverner and Emilio Vivancos and Ana Garcia-Fornes},
title={Comparative Analysis of the Efficacy in the Classification of Cognitive Distortions Using LLMs},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: EAA},
year={2025},
pages={957-965},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013399200003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 1: EAA
TI - Comparative Analysis of the Efficacy in the Classification of Cognitive Distortions Using LLMs
SN - 978-989-758-737-5
AU - Pico A.
AU - Taverner J.
AU - Vivancos E.
AU - Garcia-Fornes A.
PY - 2025
SP - 957
EP - 965
DO - 10.5220/0013399200003890
PB - SciTePress