Supporting Automated Documentation Updates in Continuous Software Development with Large Language Models

Henok Birru, Antonio Cicchetti, Malvina Latifaj

2025

Abstract

Nowadays software is ubiquitous in our society, making its role increasingly mission critical. Software applications need to be continuously maintained and evolved to keep up with the pace of market demands and emerging issues. Continuous Software Development (CSD) processes are an effective technological countermeasure to the mentioned evolutionary pressures: practices like DevOps leverage advanced automation mechanisms to streamline the application life-cycle. In this context, while handling the application development and implementation is adequately investigated, managing the continuous refinement of the corresponding documentation is a largely overlooked issue. Maintaining accurate technical documentation in CSD is challenging and time-consuming because the frequent software changes require continuous updates and such a task is handled manually. Therefore, this work investigates the automation of documentation updates in correspondence with code changes. In particular, we present CodeDocSync, an approach that uses Large Language Models (LLMs) to automate the updating of technical documentation in response to source code changes. The approach is developed to assist technical writers by summarizing code changes, retrieving updated content, and allowing follow-up questions via a chat interface. The approach has been applied to an industrial scenario and has been evaluated by using a set of well-known predefined metrics: contextual relevancy, answer relevancy, and faithfulness. These evaluations are performed for the retriever and generator components, using different LLMs, embedding models, temperature settings, and top-k values. Our solution achieves an average answer relevancy score of approximately 0.86 with Ope-nAI’s gpt-3.5-turbo and text-embedding-3-large. With an emotion prompting technique, this score increases to 0.94, testifying the viability of automation support for continuous technical documentation updates.

Paper Citation

in Harvard Style

Birru H., Cicchetti A. and Latifaj M. (2025). Supporting Automated Documentation Updates in Continuous Software Development with Large Language Models. In Proceedings of the 20th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE; ISBN 978-989-758-742-9, SciTePress, pages 92-106. DOI: 10.5220/0013286800003928

in Bibtex Style

@conference{enase25,
author={Henok Birru and Antonio Cicchetti and Malvina Latifaj},
title={Supporting Automated Documentation Updates in Continuous Software Development with Large Language Models},
booktitle={Proceedings of the 20th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE},
year={2025},
pages={92-106},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013286800003928},
isbn={978-989-758-742-9},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE
TI - Supporting Automated Documentation Updates in Continuous Software Development with Large Language Models
SN - 978-989-758-742-9
AU - Birru H.
AU - Cicchetti A.
AU - Latifaj M.
PY - 2025
SP - 92
EP - 106
DO - 10.5220/0013286800003928
PB - SciTePress