Decoding Persuasiveness in Eloquence Competitions: An Investigation into the LLM’s Ability to Assess Public Speaking

Alisa Barkar, Mathieu Chollet, Mathieu Chollet, Matthieu Labeau, Beatrice Biancardi, Chloe Clavel

2025

Abstract

The increasing importance of public speaking (PS) skills has fueled the development of automated assessment systems, yet the integration of large language models (LLMs) in this domain remains underexplored. This study investigates the application of LLMs for assessing PS by predicting persuasiveness. We propose a novel framework where LLMs evaluate criteria derived from educational literature and feedback from PS coaches, offering new interpretable textual features. We demonstrate that persuasiveness predictions of a regression model with the new features achieve a Root Mean Squared Error (RMSE) of 0.6, underperforming approach with hand-crafted lexical features (RMSE 0.51) and outperforming direct zero-shot LLM persuasiveness predictions (RMSE of 0.8). Furthermore, we find that only LLM-evaluated criteria of language level is predictable from lexical features (F1-score of 0.56), disapproving relations between these features. Based on our findings, we criticise the abilities of LLMs to analyze PS accurately. To ensure reproducibility and adaptability to emerging models, all source code and materials are publicly available on GitHub.

Paper Citation

in Harvard Style

Barkar A., Chollet M., Labeau M., Biancardi B. and Clavel C. (2025). Decoding Persuasiveness in Eloquence Competitions: An Investigation into the LLM’s Ability to Assess Public Speaking. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 538-546. DOI: 10.5220/0013158400003890

in Bibtex Style

@conference{icaart25,
author={Alisa Barkar and Mathieu Chollet and Matthieu Labeau and Beatrice Biancardi and Chloe Clavel},
title={Decoding Persuasiveness in Eloquence Competitions: An Investigation into the LLM’s Ability to Assess Public Speaking},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2025},
pages={538-546},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013158400003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - Decoding Persuasiveness in Eloquence Competitions: An Investigation into the LLM’s Ability to Assess Public Speaking
SN - 978-989-758-737-5
AU - Barkar A.
AU - Chollet M.
AU - Labeau M.
AU - Biancardi B.
AU - Clavel C.
PY - 2025
SP - 538
EP - 546
DO - 10.5220/0013158400003890
PB - SciTePress