Authors:
Yasin Sazid
1
;
Sharmista Kuri
1
;
Kazi Ahmed
2
and
Abdus Satter
1
Affiliations:
1
Institute of Information Technology, University of Dhaka, Dhaka, Bangladesh
;
2
Computer Science Department, University of New Mexico, Albuquerque, New Mexico, U.S.A.
Keyword(s):
Commit Classification, Commit Message, Maintenance Activity, Large Language Models, GPT, In-Context Learning.
Abstract:
Classifying software changes, i.e., commits into maintenance activities enables improved decision-making in software maintenance, thereby decreasing maintenance costs. Commonly, researchers have tried commit classification using keyword-based analysis of commit messages. Source code changes and density data have also been used for this purpose. Recent works have leveraged contextual semantic analysis of commit messages using pre-trained language models. But these approaches mostly depend on training data, making their ability to generalize a matter of concern. In this study, we explore the possibility of using in-context learning capabilities of large language models in commit classification. In-context learning does not require training data, making our approach less prone to data overfitting and more generalized. Experimental results using GPT-3 achieves a highest accuracy of 75.7% and kappa of 61.7%. It is similar to performances of other baseline models except one, highlighting t
he applicability of in-context learning in commit classification.
(More)