Towards Computational Performance Engineering for Unsupervised Concept Drift Detection: Complexities, Benchmarking, Performance Analysis

Elias Werner, Elias Werner, Nishant Kumar, Nishant Kumar, Matthias Lieber, Matthias Lieber, Sunna Torge, Sunna Torge, Stefan Gumhold, Stefan Gumhold, Stefan Gumhold, Wolfgang E. Nagel, Wolfgang E. Nagel, Wolfgang E. Nagel

2024

Abstract

Concept drift detection is crucial for many AI systems to ensure the system’s reliability. These systems often have to deal with large amounts of data or react in real-time. Thus, drift detectors must meet computational requirements or constraints with a comprehensive performance evaluation. However, so far, the focus of developing drift detectors is on inference quality, e.g. accuracy, but not on computational performance, such as runtime. Many of the previous works consider computational performance only as a secondary objective and do not have a benchmark for such evaluation. Hence, we propose and explain performance engineering for unsupervised concept drift detection that reflects on computational complexities, benchmarking, and performance analysis. We provide the computational complexities of existing unsupervised drift detectors and discuss why further computational performance investigations are required. Hence, we state and substantiate the aspects of a benchmark for unsupervised drift detection reflecting on inference quality and computational performance. Furthermore, we demonstrate performance analysis practices that have proven their effectiveness in High-Performance Computing, by tracing two drift detectors and displaying their performance data.

Paper Citation

in Harvard Style

Werner E., Kumar N., Lieber M., Torge S., Gumhold S. and E. Nagel W. (2024). Towards Computational Performance Engineering for Unsupervised Concept Drift Detection: Complexities, Benchmarking, Performance Analysis. In Proceedings of the 13th International Conference on Data Science, Technology and Applications - Volume 1: DATA; ISBN 978-989-758-707-8, SciTePress, pages 318-329. DOI: 10.5220/0012758600003756

in Bibtex Style

@conference{data24,
author={Elias Werner and Nishant Kumar and Matthias Lieber and Sunna Torge and Stefan Gumhold and Wolfgang E. Nagel},
title={Towards Computational Performance Engineering for Unsupervised Concept Drift Detection: Complexities, Benchmarking, Performance Analysis},
booktitle={Proceedings of the 13th International Conference on Data Science, Technology and Applications - Volume 1: DATA},
year={2024},
pages={318-329},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012758600003756},
isbn={978-989-758-707-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Data Science, Technology and Applications - Volume 1: DATA
TI - Towards Computational Performance Engineering for Unsupervised Concept Drift Detection: Complexities, Benchmarking, Performance Analysis
SN - 978-989-758-707-8
AU - Werner E.
AU - Kumar N.
AU - Lieber M.
AU - Torge S.
AU - Gumhold S.
AU - E. Nagel W.
PY - 2024
SP - 318
EP - 329
DO - 10.5220/0012758600003756
PB - SciTePress