How have Software Engineering Researchers been Measuring Software Productivity? - A Systematic Mapping Study

Edson Oliveira, Davi Viana, Marco Cristo, Tayana Conte

Abstract

Context: productivity has been a recurring topic, and despite its importance, researchers have not yet reached a consensus on how to properly measure productivity in software engineering. Aim: to investigate and better understand how software productivity researchers are using software productivity metrics. Method: we performed a systematic mapping study on publications regarding software productivity, extracting how software engineering researchers are measuring software productivity. Results: In a total of 91 software productivity metrics were extracted. The obtained results show that researchers apply these productivity metrics mainly focusing on software projects and developers, and these productivity metrics are predominantly composed by Lines of Code (LOC), Time and Effort measures. Conclusion: although there is no consensus, our results shows that single ratio metrics, such as LOC/Effort, for software projects, and LOC/Time, for software developers, are a tendency adopted by researchers to measure productivity.

References

  1. Amrit, C., Daneva, M. & Damian, D., 2014. Human factors in software development: On its underlying theories and the value of learning from related disciplines. A guest editorial introduction to the special issue. Information and Software Technology, 56(12), pp.1537-1542.
  2. Aquino Junior, G.S. de & Meira, S.R.L., 2009. Towards effective productivity measurement in software projects. In Proceedings of the 4th International Conference on Software Engineering Advance. IEEE, pp. 241-249.
  3. Barb, A.S., Neill, J., Sangwan, S., Piovoso, J., 2014. A statistical study of the relevance of lines of code measures in software projects. Innovations in Systems and Software Engineering, pp.243-260.
  4. Basili, V.R. & Rombach, H.D., 1988. Tame Project: Towards Improvement-Oriented Software Environments. IEEE Transactions on Software Engineering, 14(6), pp.758-773.
  5. Boehm, 1987. Improving Software Productivity. Computer, 20(9), pp.43-57.
  6. Cheikhi, L., Al-Qutaish, R.E. & Idri, A., 2012. Software Productivity: Harmonization in ISO/IEEE Software Engineering Standards. Journal of Software, 7(2), pp.462-470.
  7. Cohen, J., 1960. A coefficient of agreement of nominal scales. Educational and Psychological Measurement, 20(1), pp.37-46.
  8. DeMarco, T., 1986. Controlling Software Projects: Management, Measurement, and Estimation, Upper Saddle River, NJ: Prentice Hall PTR.
  9. Dixon-Woods, Agarwal, M., Jones, S., Young, D., B., Sutton, A., 2005. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy, 10(1), pp.45-53.
  10. Fenton, N.E. & Pfleeger, S.L., 1998. Software Metrics: A Rigorous and Practical Approach 2nd ed., Boston, MA, USA: PWS Publishing Co.
  11. Hernández-López, A., Colomo-Palacios, R., GarcíaCrespo, A., Cabezas-Isla, F., 2011. Software Engineering Productivity: Concepts, Issues and Challenges. International Journal of Information Technology Project Management, 2(1), pp.37-47.
  12. Hernández-López, A., Colomo-Palacios, R. & GarcíaCrespo, Á., 2013. Software Engineering Job Productivity-a Systematic Review. International Journal of Software Engineering and Knowledge Engineering, 23(3), pp.387-406.
  13. Jalali, S. & Wohlin, C., 2012. Systematic literature studies: Database Searches vs. Backward Snowballing. In Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement - ESEM 7812. New York, New York, USA: ACM Press, p. 29.
  14. Kitchenham, B. & Charters, S., 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering, Keele, UK.
  15. Landis, J.R. & Koch, G.G., 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics, 33(1), p.159.
  16. Meyer, A.N., Fritz, T., Murphy, G. C., Zimmermann, T., 2014. Software developers' perceptions of productivity. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York, New York, USA: ACM Press, pp. 19-29.
  17. Mockus, A., 2009. Succession: Measuring transfer of code and developer productivity. In Proceedings of the 2009 IEEE 31st International Conference on Software Engineering. Vancouver, BC: IEEE, pp. 67-77.
  18. Oliveira, E., Viana, D., Cristo, M. & Conte, T., 2017. “A Systematic Mapping on Productivity Metrics in Software Development and Maintenance”, TR-USES2017-0002. Available online at: http://uses.icomp.ufam.edu.br/relatorios-tecnico.
  19. Petersen, K., Feldt, R., Mujtaba, S., Mattsson, M., 2008. Systematic mapping studies in software engineering. EASE'08 Proceedings of the 12th international conference on Evaluation and Assessment in Software Engineering, pp.68-77.
  20. Petersen, K. & Wohlin, C., 2009. Context in industrial software engineering research. 2009 3rd International Symposium on Empirical Software Engineering and Measurement, ESEM 2009, pp.401-404.
  21. Petersen, K., 2011. Measuring and predicting software productivity: A systematic map and review. Information and Software Technology, 53(4), pp.317- 343.
  22. Petticrew, M. & Roberts, H., 2006. Systematic Reviews in the Social Sciences: A Practical Guide,
  23. Siegel, S. & Castellan, N.J., 1988. Nonparametric statistics for the behavioral sciences (2nd ed.),
  24. Trendowicz, A. & Münch, J., 2009. Factors Influencing Software Development Productivity - State of the Art and Industrial Experiences. Advances in Computers, 77(9), pp.185-241.
  25. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M., Regnell, B., Wesslén, A., 2012. Experimentation in Software Engineering, Berlin, Heidelberg: Springer Publishing Company, Incorporated.
Download


Paper Citation


in Harvard Style

Oliveira E., Viana D., Cristo M. and Conte T. (2017). How have Software Engineering Researchers been Measuring Software Productivity? - A Systematic Mapping Study . In Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 2: ICEIS, ISBN 978-989-758-248-6, pages 76-87. DOI: 10.5220/0006314400760087


in Bibtex Style

@conference{iceis17,
author={Edson Oliveira and Davi Viana and Marco Cristo and Tayana Conte},
title={How have Software Engineering Researchers been Measuring Software Productivity? - A Systematic Mapping Study},
booktitle={Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 2: ICEIS,},
year={2017},
pages={76-87},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006314400760087},
isbn={978-989-758-248-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 2: ICEIS,
TI - How have Software Engineering Researchers been Measuring Software Productivity? - A Systematic Mapping Study
SN - 978-989-758-248-6
AU - Oliveira E.
AU - Viana D.
AU - Cristo M.
AU - Conte T.
PY - 2017
SP - 76
EP - 87
DO - 10.5220/0006314400760087