The Effect of Covariate Shift and Network Training on Out-of-Distribution Detection

Simon Mariani, Sander Klomp, Rob Romijnders, Peter H. N. de With

2023

Abstract

The field of Out-of-Distribution (OOD) detection aims to separate OOD data from in-distribution (ID) data in order to make safe predictions. With the increasing application of Convolutional Neural Networks (CNNs) in sensitive environments such as autonomous driving and security, this field is bound to become indispensable in the future. Although the OOD detection field has made some progress in recent years, a fundamental understanding of the underlying phenomena enabling the separation of datasets remains lacking. We find that the OOD detection relies heavily on the covariate shift of the data and not so much on the semantic shift, i.e. a CNN does not carry explicit semantic information and relies solely on differences in features. Although these features can be affected by the underlying semantics, this relation does not seem strong enough to rely on. Conversely, we found that since the CNN training setup determines what features are learned, that it is an important factor for the OOD performance. We found that variations in the model training can lead to an increase or decrease in the OOD detection performance. Through this insight, we obtain an increase in OOD detection performance on the common OOD detection benchmarks by changing the training procedure and using the simple Maximum Softmax Probability (MSP) model introduced by (Hendrycks and Gimpel, 2016). We hope to inspire others to look more closely into the fundamental principles underlying the separation of two datasets. The code for reproducing our results can be found at https://github.com/SimonMariani/OOD- detection.

Paper Citation

in Harvard Style

Mariani S., Klomp S., Romijnders R. and H. N. de With P. (2023). The Effect of Covariate Shift and Network Training on Out-of-Distribution Detection. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 723-730. DOI: 10.5220/0011725900003417

in Bibtex Style

@conference{visapp23,
author={Simon Mariani and Sander Klomp and Rob Romijnders and Peter H. N. de With},
title={The Effect of Covariate Shift and Network Training on Out-of-Distribution Detection},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP},
year={2023},
pages={723-730},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011725900003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP
TI - The Effect of Covariate Shift and Network Training on Out-of-Distribution Detection
SN - 978-989-758-634-7
AU - Mariani S.
AU - Klomp S.
AU - Romijnders R.
AU - H. N. de With P.
PY - 2023
SP - 723
EP - 730
DO - 10.5220/0011725900003417
PB - SciTePress