Defining and Evaluating Software Project Success Indicators - A GQM-based Case Study

Luigi Lavazza, Enrico Frumento, Riccardo Mazza

2015

Abstract

KPI (Key Process Indicators) and success indicators are often defined in a rather generic and imprecise manner. This happens because they are defined very early in the project’s life, when little details about the project are known, or simply because the definition does not follow a systematic and effective methodology. We need to precisely define KPI and project success indicators, guarantee that the data upon which they are based can be effectively and efficiently measured, and assure that the computed indicators are adequate with respect to project objectives, and represent the viewpoints of all the involved stakeholders. A complete and coherent process for managing KPI and success indicators lifecycle –instrumented with specific techniques and tools, including the Goal/Question/Metrics (GQM) method for the definition of measures and the R statistic language and environment for analyzing data and computing indicators– was applied in the evaluation of the European research project MUSES. The MUSES case study shows that the proposed process provides an easy and well supported path to the definition and implementation of effective KPI and project success indicators.

References

  1. MUSES, 2014. https://www.musesproject.eu/
  2. Basili, V., Rombach, H.D., 1988. The TAME project: towards improvement-oriented software environments, IEEE Trans. on Software Engineering, vol. 14, n. 6.
  3. Basili, V., Weiss, D., 1984. A methodology for collecting valid software engineering data, IEEE Trans. on Software Engineering, vol. 10, n. 6.
  4. Basili, V., Caldiera, G., Rombach, H.D., 1994. Goal/Question/Metric Paradigm in Encyclopedia of Software Engineering, vol. 1: Wiley, 1994.
  5. Basili, V., Caldiera, G., Rombach, H.D., 1994. Experience factory in Encyclopedia of Software Engineering, vol. 1, J. C. Marciniak, Ed.: John Wiley & Sons, 1994.
  6. Fuggetta, A.. Lavazza, L., Morasca, S., Cinti,S., Oldano, G., Orazi, E., 1998. Applying G/Q/M in an Industrial Software Factory, ACM Trans. on Software Engineering and Methodology, vol. 7, n. 4.
  7. Birk, A., van Solingen, R., Jarvinen, J. 1998. Business Impact, Benefit, and Cost of Applying GQM in Industry: An In-Depth, Long-Term Investigation at Schlumberger RPS, in 5th Int. Symp. on Software Metrics (Metrics'98).
  8. van Solingen, R., Berghout, E. 2001. Integrating GoalOriented Measurement in Industrial Software Engineering: Industrial Experiences with and Additions to the Goal/Question/Metric Method (GQM)”, in 7th Int. Software Metrics Symposium (METRICS 2001).
  9. Lavazza, L., 2011. Multi-Scope Evaluation of Public Administration Initiatives in Process Automation, in 5th European Conf. on Information Management and Evaluation (ECIME 2011).
  10. Lavazza, L., Mauri, M. 2006. Software Process Measurement in the Real World: Dealing with Operating Constraints. In Software Process Workshop SPW/Prosim 2006. Springer LNCS Vol. 3966/2006.
  11. Lavazza L., Barresi, G. 2005. Automated Support for Process-aware Definition and Execution of Measurement Plans, in Int. Conf. on Software Engineering (ICSE 2005).
  12. Bangor, A., Kortum, P., Miller J. 2009. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies.
  13. Brooke J. 1996. SUS: A “quick and dirty” usability scale. In P. Jordan, B. Thomas, & B. Weerdemeester (Eds.), Usability Evaluation in Industry.
  14. Lewis J.R., Sauro, J. 2009. The Factor Structure of the System Usability Scale. In 1st Int. Conf. on Human Centered Design (HCD 09).
  15. Venkatesh, V., Bala, H. 2008. Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences, vol. 39, n. 2.
  16. Likert, R. 1932. Technique for the measure of attitudes Arch. Psycho, vol. 22 n. 140.
  17. Ihaka, R., Gentleman, R. 1996. R: a language for data analysis and graphics. Journal of computational and graphical statistics, vol. 5, n. 3.
  18. Ripley, B., Lapsley, M. 2014, Package RODBC - ODBC Database Access.
  19. Münch, J., Heidrich, J. 2004. Software project control centers: concepts and approaches. Journal of Systems and Software, vol. 70 n.1.
  20. Tsunoda, M., Matsumura, T., Matsumoto, K.I. 2010. Modeling Software Project Monitoring with Stakeholders, in 9th Int. Conf. on Computer and Information Science (ICIS 2010).
  21. Basili, V., Lindvall, M., Regardie, M., Seaman, C., Heidrich, J., Munch, J., Rombach, D. and Trendowicz, A. 2010. Linking Software Development and Business Strategy through Measurement, Computer, vol.43, n.4.
  22. Souza Cardoso, E. C., 2013. Towards a Methodology for Goal-Oriented Enterprise Management. In Enterprise Distributed Object Computing Conference Workshops (EDOCW 2013).
  23. Nicho, M., Cusack, B., 2007. A metrics generation model for measuring the control objectives of information systems audit. In 40th Annual Hawaii Int. Conf. on System Sciences (HICSS 2007).
  24. Offen, R. J., Jeffery, R. 1997. Establishing software measurement programs. IEEE Software vol. 14 n. 2.
  25. Birk, A., Hamann, D., Pfahl, D., Järvinen, J., Oivo, M., Vierimaa, M., van Solingen, R. 1999. The Role of GQM in the PROFES Improvement Methodology.
  26. Park, R. E., Goethert, W. B., Florac, W. A. 1996. GoalDriven Software Measurement. A Guidebook. Software Engineering Inst., Carnegie-Mellon Univ.
  27. Basili, V. 1993. Applying the Goal/Question/Metric paradigm in the experience factory. In 10th Annual CSR Workshop, Application of Software Metrics and Quality Assurance in Industry.
  28. ISACA, 2012. COBIT 5: A Business Framework for the Governance and Management of Enterprise IT.
Download


Paper Citation


in Harvard Style

Lavazza L., Frumento E. and Mazza R. (2015). Defining and Evaluating Software Project Success Indicators - A GQM-based Case Study . In Proceedings of the 10th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2015) ISBN 978-989-758-114-4, pages 105-116. DOI: 10.5220/0005517301050116


in Bibtex Style

@conference{icsoft-ea15,
author={Luigi Lavazza and Enrico Frumento and Riccardo Mazza},
title={Defining and Evaluating Software Project Success Indicators - A GQM-based Case Study},
booktitle={Proceedings of the 10th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2015)},
year={2015},
pages={105-116},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005517301050116},
isbn={978-989-758-114-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Software Engineering and Applications - Volume 1: ICSOFT-EA, (ICSOFT 2015)
TI - Defining and Evaluating Software Project Success Indicators - A GQM-based Case Study
SN - 978-989-758-114-4
AU - Lavazza L.
AU - Frumento E.
AU - Mazza R.
PY - 2015
SP - 105
EP - 116
DO - 10.5220/0005517301050116