Runtime Verification for Deep Learning Systems

Birk Torpmann-Hagen, Michael Riegler, Pål Halvorsen, Dag Johansen

2025

Abstract

Deep Neural Networks are being utilized in increasingly numerous software systems and across a wide range of data modalities. While this affords many opportunities, recent work has also shown that deep learning systems often fail to perform up to specification in deployment scenarios, despite initial tests often indicating excellent results. This disparity can be attributed to shifts in the nature of the input data at deployment time and the infeasibility of generating test cases that sufficiently represent data that have undergone such shifts. To address this, we leverage recent advances in uncertainty quantification for deep neural networks and outline a framework for developing runtime verification support for deep learning systems. This increases the resilience of the system in deployment conditions and provides an increased degree of transparency with respect to the system’s overall real-world performance. As part of our framework, we review and systematize disparate work on quantitative methods of detecting and characterizing various failure modes in deep learning systems, which we in turn consolidate into a comprehensive framework for the implementation of flexible runtime monitors. Our framework is based on requirements analysis, and includes support for multimedia systems and online learning. As the methods we review have already been empirically verified in their respective works, we illustrate the potential of our framework through a proof-of-concept multimedia diagnostic support system architecture that utilizes our framework. Finally, we suggest directions for future research into more advanced instrumentation methods and various framework extensions. Overall, we envision that runtime verification may endow multimedia deep learning systems with the necessary resilience required for deployment in real-world applications.

Paper Citation

in Harvard Style

Torpmann-Hagen B., Riegler M., Halvorsen P. and Johansen D. (2025). Runtime Verification for Deep Learning Systems. In Proceedings of the 20th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE; ISBN 978-989-758-742-9, SciTePress, pages 367-377. DOI: 10.5220/0013195500003928

in Bibtex Style

@conference{enase25,
author={Birk Torpmann-Hagen and Michael Riegler and Pål Halvorsen and Dag Johansen},
title={Runtime Verification for Deep Learning Systems},
booktitle={Proceedings of the 20th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE},
year={2025},
pages={367-377},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013195500003928},
isbn={978-989-758-742-9},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE
TI - Runtime Verification for Deep Learning Systems
SN - 978-989-758-742-9
AU - Torpmann-Hagen B.
AU - Riegler M.
AU - Halvorsen P.
AU - Johansen D.
PY - 2025
SP - 367
EP - 377
DO - 10.5220/0013195500003928
PB - SciTePress