Investigating the Use of a Contextualized Vocabulary in the Identification of Technical Debt: A Controlled Experiment

Mário André de Freitas Farias, José Amancio Santos, André Batista da Silva, Marcos Kalinowski, Manoel Mendonça, Rodrigo Oliveira Spínola

2016

Abstract

In order to effectively manage technical debt (TD), a set of indicators has been used by automated approaches to identify TD items. However, some debt may not be directly identified using only metrics collected from the source code. CVM-TD is a model to support the identification of technical debt by considering the developer point of view when identifying TD through code comment analysis. In this paper, we analyze the use of CVM-TD with the purpose of characterizing factors that affect the accuracy of the identification of TD. We performed a controlled experiment investigating the accuracy of CVM-TD and the influence of English skills and developer experience factors. The results indicated that CVM-TD provided promising results considering the accuracy values. English reading skills have an impact on the TD detection process. We could not conclude that the experience level affects this process. Finally, we also observed that many comments suggested by CVM-TD were considered good indicators of TD. The results motivate us continuing to explore code comments in the context of TD identification process in order to improve CVM-TD.

References

  1. Alves, N.S.R. et al., 2016. Identification and Management of Technical Debt: A Systematic Mapping Study. Information and Software Technology, pp.100-121.
  2. Alves, N.S.R. et al., 2014. Towards an Ontology of Terms on Technical Debt. 6th MTD. pp. 1-7.
  3. Wohlin, C and Runeson, M.H., 2000. Experimentation in Software Engineering: an introduction, Kluwer Academic Publishers Norwell.
  4. Ernst, N.A. et al., 2015. Measure It?? Manage It?? Ignore It?? Software Practitioners and Technical Debt. 10th Joint Meeting on Found. of Soft. Engineering. ACM.
  5. Farias, M. et al., 2015. A Contextualized Vocabulary Model for Identifying Technical Debt on Code Comments. 7th MTD. pp. 25-32.
  6. Finn, R.H., 1970. A Note on Estimating the Reliability of Categorical Data. Educational and Psychological Measurement, pp.71-76.
  7. Guo, Y. et al., 2014. Exploring the costs of technical debt management - a case study. ESE, 1, pp.1-24.
  8. Host, M., Wohlin, C. and Thelin, T., 2005. Experimental context classification: incentives and experience of subjects. 27th ICSE, pp.470-478.
  9. Izurieta, C. et al., 2012. Organizing the technical debt landscape. 2012 3rd MTD, pp.23-26.
  10. Cohen, J. 1988. Statistical power analysis for the behavioral sciences. 2 edition. L. Erlbaum, ed.,
  11. Kruchten, P. et al., I., 2012. Technical debt: From metaphor to theory and practice. IEEE, pp.18-21.
  12. Landis, J.R. and Koch, G.G., 1977. The measurement of observer agreement for categorical data. Biometrics, pp.159-174.
  13. Lemos, O. a L. et al., 2014. Thesaurus-Based Automatic Query Expansion for Interface-Driven Code Search Categories and Subject Descriptors, pp.212-221.
  14. Li, Z. et al., 2014. A systematic mapping study on technical debt. Journal of Syst. Soft. 101, pp.193-220.
  15. Maalej, W. and Happel, H.-J., 2010. Can development work describe itself? 7th MSR, pp.191-200.
  16. Maldonado, E.S. and Shihab, E., 2015. Detecting and Quantifying Different Types of Self-Admitted Technical Debt. In 7th MTD. pp. 9-15.
  17. Mendes, T. et al., 2015. VisMinerTD - An Open Source Tool to Support the Monitoring of the Technical Debt Evolution using Software Visualization. 17th ICEIS.
  18. Potdar, A. and Shihab, E., 2014. An Exploratory Study on Self-Admitted Technical Debt. ICSME, pp. 91-100.
  19. Salman, I., 2015. Are Students Representatives of Professionals in Software Engineering Experiments? 37th ICSE. IEEE Press, 2015.
  20. Santos, J.A.M., et al., 2014. The problem of conceptualization in god class detection?: agreement , strategies and decision drivers. Journal of Software Engineering Research and Development, (2), pp.1-33.
  21. Shull, F., Singer, J. and Sjoberg, D., 2008. Guide to Advanced Empirical Software Engineering, Springer.
  22. Snedecor, G.W. and Cochran, W.G., 1967. Statistical Methods. Ames.
  23. Spínola, R. et al., 2013. Investigating Technical Debt Folklore. 5th MTD, pp.1-7.
  24. Storey, M. et al., 2008. TODO or To Bug?: Exploring How Task Annotations Play a Role in the Work Practices of Software Developers. ICSE. pp. 251-260.
  25. Zazworka, N. et al., 2013. A case study on effectively identifying technical debt. 17thEASE. ACM, pp.42-47.
Download


Paper Citation


in Harvard Style

Farias M., Santos J., da Silva A., Kalinowski M., Mendonça M. and Spínola R. (2016). Investigating the Use of a Contextualized Vocabulary in the Identification of Technical Debt: A Controlled Experiment . In Proceedings of the 18th International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-758-187-8, pages 369-378. DOI: 10.5220/0005914503690378


in Bibtex Style

@conference{iceis16,
author={Mário André de Freitas Farias and José Amancio Santos and André Batista da Silva and Marcos Kalinowski and Manoel Mendonça and Rodrigo Oliveira Spínola},
title={Investigating the Use of a Contextualized Vocabulary in the Identification of Technical Debt: A Controlled Experiment},
booktitle={Proceedings of the 18th International Conference on Enterprise Information Systems - Volume 1: ICEIS,},
year={2016},
pages={369-378},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005914503690378},
isbn={978-989-758-187-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 18th International Conference on Enterprise Information Systems - Volume 1: ICEIS,
TI - Investigating the Use of a Contextualized Vocabulary in the Identification of Technical Debt: A Controlled Experiment
SN - 978-989-758-187-8
AU - Farias M.
AU - Santos J.
AU - da Silva A.
AU - Kalinowski M.
AU - Mendonça M.
AU - Spínola R.
PY - 2016
SP - 369
EP - 378
DO - 10.5220/0005914503690378