Exploring the Use of ChatGPT for the Generation of User Story Based Test Cases: An Experimental Study

Felipe Manzoni, Felipe Manzoni, Rávella Rodrigues, Ana Carolina Oran Rocha

2025

Abstract

CONTEXT: The rapid advancement of Artificial Intelligence (AI) technologies has introduced new tools and methodologies in software engineering, particularly in test case generation. Traditional methods for generating test cases are often time-consuming and rely on manual input, limiting efficiency and coverage. The ChatGPT 3.5 model, developed by OpenAI, represents a novel approach to automating this process, potentially transforming software testing. OBJECTIVE: This article aims to explore the application of ChatGPT 3.5 in generating test cases based on user stories from a course in software engineering, evaluating the effectiveness, user acceptance, and challenges associated with its implementation. METHOD: The study involved generating test cases using ChatGPT 3.5 and executed by students from the Practice in Software Engineering (PES) course at the Federal University of Amazonas (UFAM) collecting data through surveys and qualitative feedback, focusing on TAM model perceptions and students’ self-perceptions. RESULTS and CONCLUSIONS: Results indicate a generally positive reception of ChatGPT 3.5 for the objective above, praising it for enhancing several aspects of TC creation, which resulted in high intention of future use and perception of value. However, some challenges have been raised, meaning users should validate and review generated results. Furthermore, results highlight the importance of integrating AI tools while keeping human expertise to maximize their effectiveness.

Paper Citation

in Harvard Style

Manzoni F., Rodrigues R. and Rocha A. (2025). Exploring the Use of ChatGPT for the Generation of User Story Based Test Cases: An Experimental Study. In Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS; ISBN 978-989-758-749-8, SciTePress, pages 292-299. DOI: 10.5220/0013398700003929

in Bibtex Style

@conference{iceis25,
author={Felipe Manzoni and Rávella Rodrigues and Ana Rocha},
title={Exploring the Use of ChatGPT for the Generation of User Story Based Test Cases: An Experimental Study},
booktitle={Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS},
year={2025},
pages={292-299},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013398700003929},
isbn={978-989-758-749-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS
TI - Exploring the Use of ChatGPT for the Generation of User Story Based Test Cases: An Experimental Study
SN - 978-989-758-749-8
AU - Manzoni F.
AU - Rodrigues R.
AU - Rocha A.
PY - 2025
SP - 292
EP - 299
DO - 10.5220/0013398700003929
PB - SciTePress