User Stories: Does ChatGPT Do It Better?

Reine Santos, Gabriel Freitas, Igor Steinmacher, Tayana Conte, Ana Oran, Bruno Gadelha

2025

Abstract

In agile software development, user stories play a central role in defining system requirements, fostering communication, and guiding development efforts. Despite their importance, they are often poorly written, exhibiting quality defects that hinder project outcomes and reduce team efficiency. Manual methods for creating user stories are time-consuming and prone to errors and inconsistencies. Advancements in Large Language Models (LLMs), such as ChatGPT, present a promising avenue for automating and improving this process. This research explores whether user stories generated by ChatGPT, using prompting techniques, achieve higher quality than those created manually by humans. User stories were assessed using the Quality User Story (QUS) framework. We conducted two empirical studies to address this. The first study compared manually created user stories with those generated by ChatGPT through free-form prompt. This study involved 30 participants and found no statistically significant difference between the two methods. The second study compared free-form prompt with meta-few-shot prompt, demonstrating that the latter outperformed both, achieving higher consistency and semantic quality with an efficiency calculated based on the success rate of 88.57%. These findings highlight the potential of LLMs with prompting techniques to enhance user story generation, offering a reliable and effective alternative to traditional methods.

Paper Citation

in Harvard Style

Santos R., Freitas G., Steinmacher I., Conte T., Oran A. and Gadelha B. (2025). User Stories: Does ChatGPT Do It Better?. In Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS; ISBN 978-989-758-749-8, SciTePress, pages 47-58. DOI: 10.5220/0013365500003929

in Bibtex Style

@conference{iceis25,
author={Reine Santos and Gabriel Freitas and Igor Steinmacher and Tayana Conte and Ana Oran and Bruno Gadelha},
title={User Stories: Does ChatGPT Do It Better?},
booktitle={Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS},
year={2025},
pages={47-58},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013365500003929},
isbn={978-989-758-749-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS
TI - User Stories: Does ChatGPT Do It Better?
SN - 978-989-758-749-8
AU - Santos R.
AU - Freitas G.
AU - Steinmacher I.
AU - Conte T.
AU - Oran A.
AU - Gadelha B.
PY - 2025
SP - 47
EP - 58
DO - 10.5220/0013365500003929
PB - SciTePress