Study of Uncertainty Quantification Using Multi-Label ECG in Deep Learning Models

Raquel Simão, Raquel Simão, Marília Barandas, Marília Barandas, David Belo, Hugo Gamboa, Hugo Gamboa

2023

Abstract

Machine Learning (ML) models can predict diseases with noteworthy results. However, when implemented, their generalization are compromised, resulting in lower performances and render healthcare professionals more susceptible into delivering erroneous diagnostics. This study focuses on the use of uncertainty measures to abstain from classifying samples and use the rejected samples as a selection criterion for active learning. For the multi-label classification of cardiac arrhythmias different methods for uncertainty quantification were compared using three Deep Learning (DL) models: a single model and two pseudoensemble models using Monte-Carlo (MC) Dropout and Deep Ensemble (DE) techniques. When tested with an external dataset, the models’ performances dropped from a F1-Score of 96% to 70%, indicating the possibility of dataset shift. The uncertainty measures for classification with rejection resulted in an increase of the rejection rate from 10% in the training set to a range between 30% to 50% on the external dataset. For the active learning approach, 10% of the highest uncertainty samples were used to retrain the models and their performance increased by almost 5%. Although there are still challenges to the implementation of ML models, the results show that uncertainty quantification is a valuable method to employ in safety mechanisms under dataset shift conditions.

Paper Citation

in Harvard Style

Simão R., Barandas M., Belo D. and Gamboa H. (2023). Study of Uncertainty Quantification Using Multi-Label ECG in Deep Learning Models. In Proceedings of the 16th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2023) - Volume 4: BIOSIGNALS; ISBN 978-989-758-631-6, SciTePress, pages 252-259. DOI: 10.5220/0011680700003414

in Bibtex Style

@conference{biosignals23,
author={Raquel Simão and Marília Barandas and David Belo and Hugo Gamboa},
title={Study of Uncertainty Quantification Using Multi-Label ECG in Deep Learning Models},
booktitle={Proceedings of the 16th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2023) - Volume 4: BIOSIGNALS},
year={2023},
pages={252-259},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011680700003414},
isbn={978-989-758-631-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2023) - Volume 4: BIOSIGNALS
TI - Study of Uncertainty Quantification Using Multi-Label ECG in Deep Learning Models
SN - 978-989-758-631-6
AU - Simão R.
AU - Barandas M.
AU - Belo D.
AU - Gamboa H.
PY - 2023
SP - 252
EP - 259
DO - 10.5220/0011680700003414
PB - SciTePress