Predicting User Satisfaction in Software Projects using Machine Learning Techniques

Łukasz Radliński

2020

Abstract

User satisfaction is an important aspect of software quality. Factors of user satisfaction and its impact on project success were analysed in various studies. However, very few studies investigated the ability to predict user satisfaction. This paper presents results of such challenge. The analysis was performed with the ISBSG dataset of software projects. The target variable, satisfaction score, was defined as a sum of eight variables reflecting different aspects of user satisfaction. Twelve machine learning algorithms were used to build 40 predictive models. Each model was evaluated on 20 passes with a test subset. On average, a random forest model with missing data imputation by mode and mean achieved the best performance with the macro mean absolute error of 1.88. Four variables with the highest importance on predictions for this model are: survey respondent role, log(effort estimate), log(summary work effort), and proportion of major defects. On average 14 models performed worse than a simple baseline model. While best performing models deliver predictions with satisfactory accuracy, high variability of performance between different model variants was observed. Thus, a careful selection of model settings is required when attempting to use such model in practise.

Paper Citation

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE,
TI - Predicting User Satisfaction in Software Projects using Machine Learning Techniques
SN - 978-989-758-421-3
AU - Radliński Ł.
PY - 2020
SP - 374
EP - 381
DO - 10.5220/0009391803740381

in Harvard Style

Radliński Ł. (2020). Predicting User Satisfaction in Software Projects using Machine Learning Techniques.In Proceedings of the 15th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE, ISBN 978-989-758-421-3, pages 374-381. DOI: 10.5220/0009391803740381

in Bibtex Style

@conference{enase20,
author={Łukasz Radliński},
title={Predicting User Satisfaction in Software Projects using Machine Learning Techniques},
booktitle={Proceedings of the 15th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE,},
year={2020},
pages={374-381},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009391803740381},
isbn={978-989-758-421-3},
}