Authors:
Simon Staudinger
;
Christoph G. Schuetz
and
Michael Schrefl
Affiliation:
Institute of Business Informatics, Data and Knowledge Engineering, Johannes Kepler University Linz, Austria
Keyword(s):
Business Intelligence, Business Analytics, Decision Support Systems, Data Mining, CRISP-DM.
Abstract:
Organizations employ data mining to discover patterns in historic data. The models that are learned from the data allow analysts to make predictions about future events of interest. Different global measures, e.g., accuracy, sensitivity, and specificity, are employed to evaluate a predictive model. In order to properly assess the reliability of an individual prediction for a specific input case, global measures may not suffice. In this paper, we propose a reference process for the development of predictive analytics applications that allow analysts to better judge the reliability of individual classification results. The proposed reference process is aligned with the CRISP-DM stages and complements each stage with a number of tasks required for reliability checking. We further explain two generic approaches that assist analysts with the assessment of reliability of individual predictions, namely perturbation and local quality measures.