Acceptance Criteria Validation in Agile Projects Using AI and NLP Techniques

Ana Carla Gomes da Silva, Afonso Sales, Fabio Gomes Rocha

2025

Abstract

In agile software development, user stories and their acceptance criteria play a critical role in ensuring alignment between stakeholder expectations and system functionality. However, the manual validation of these criteria is often labor-intensive and prone to bias. This study investigates the application of Artificial Intelligence (AI) techniques, particularly Natural Language Processing (NLP) and Machine Learning (ML), to automate the analysis and validation of user stories. Using a dataset of user stories collected from academic and industry projects, we trained and evaluated four ML algorithms: Multilayer Perceptron (MLP), Support Vector Machine (SVM), Naive Bayes, and Random Forest. The models were assessed for their ability to classify acceptance criteria accurately and efficiently. Our findings demonstrate the potential of AI to enhance the validation process, achieving over 60% accuracy in certain cases, with SVM standing out as the most robust algorithm. This research highlights the transformative role of AI in improving software requirements analysis and lays the foundation for future innovations in automated validation and quality assurance in agile environments.

Paper Citation

in Harvard Style

Gomes da Silva A., Sales A. and Rocha F. (2025). Acceptance Criteria Validation in Agile Projects Using AI and NLP Techniques. In Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS; ISBN 978-989-758-749-8, SciTePress, pages 176-184. DOI: 10.5220/0013276400003929

in Bibtex Style

@conference{iceis25,
author={Ana Gomes da Silva and Afonso Sales and Fabio Rocha},
title={Acceptance Criteria Validation in Agile Projects Using AI and NLP Techniques},
booktitle={Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS},
year={2025},
pages={176-184},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013276400003929},
isbn={978-989-758-749-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 2: ICEIS
TI - Acceptance Criteria Validation in Agile Projects Using AI and NLP Techniques
SN - 978-989-758-749-8
AU - Gomes da Silva A.
AU - Sales A.
AU - Rocha F.
PY - 2025
SP - 176
EP - 184
DO - 10.5220/0013276400003929
PB - SciTePress