MA-VAE: Multi-Head Attention-Based Variational Autoencoder Approach for Anomaly Detection in Multivariate Time-Series Applied to Automotive Endurance Powertrain Testing

Lucas Correia, Lucas Correia, Jan-Christoph Goos, Philipp Klein, Thomas Bäck, Anna Kononova

2023

Abstract

A clear need for automatic anomaly detection applied to automotive testing has emerged as more and more attention is paid to the data recorded and manual evaluation by humans reaches its capacity. Such real-world data is massive, diverse, multivariate and temporal in nature, therefore requiring modelling of the testee behaviour. We propose a variational autoencoder with multi-head attention (MA-VAE), which, when trained on unlabelled data, not only provides very few false positives but also manages to detect the majority of the anomalies presented. In addition to that, the approach offers a novel way to avoid the bypass phenomenon, an undesirable behaviour investigated in literature. Lastly, the approach also introduces a new method to remap individual windows to a continuous time series. The results are presented in the context of a real-world in-dustrial data set and several experiments are undertaken to further investigate certain aspects of the proposed model. When configured properly, it is 9% of the time wrong when an anomaly is flagged and discovers 67% of the anomalies present. Also, MA-VAE has the potential to perform well with only a fraction of the training and validation subset, however, to extract it, a more sophisticated threshold estimation method is required.

Paper Citation

in Harvard Style

Correia L., Goos J., Klein P., Bäck T. and Kononova A. (2023). MA-VAE: Multi-Head Attention-Based Variational Autoencoder Approach for Anomaly Detection in Multivariate Time-Series Applied to Automotive Endurance Powertrain Testing. In Proceedings of the 15th International Joint Conference on Computational Intelligence - Volume 1: NCTA; ISBN 978-989-758-674-3, SciTePress, pages 407-418. DOI: 10.5220/0012163100003595

in Bibtex Style

@conference{ncta23,
author={Lucas Correia and Jan-Christoph Goos and Philipp Klein and Thomas Bäck and Anna Kononova},
title={MA-VAE: Multi-Head Attention-Based Variational Autoencoder Approach for Anomaly Detection in Multivariate Time-Series Applied to Automotive Endurance Powertrain Testing},
booktitle={Proceedings of the 15th International Joint Conference on Computational Intelligence - Volume 1: NCTA},
year={2023},
pages={407-418},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012163100003595},
isbn={978-989-758-674-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Joint Conference on Computational Intelligence - Volume 1: NCTA
TI - MA-VAE: Multi-Head Attention-Based Variational Autoencoder Approach for Anomaly Detection in Multivariate Time-Series Applied to Automotive Endurance Powertrain Testing
SN - 978-989-758-674-3
AU - Correia L.
AU - Goos J.
AU - Klein P.
AU - Bäck T.
AU - Kononova A.
PY - 2023
SP - 407
EP - 418
DO - 10.5220/0012163100003595
PB - SciTePress