Hallucinated Heartbeats: Anomaly-Aware Remote Pulse Estimation

Jeremy Speth, Nathan Vance, Benjamin Sporrer, Lu Niu, Patrick Flynn, Adam Czajka

2023

Abstract

Camera-based physiological monitoring, especially remote photoplethysmography (rPPG), is a promising tool for health diagnostics, and state-of-the-art pulse estimators have shown impressive performance on benchmark datasets. We argue that evaluations of modern solutions may be incomplete, as we uncover failure cases for videos without a live person, or in the presence of severe noise. We demonstrate that spatiotemporal deep learning models trained only with live samples “hallucinate” a genuine-shaped pulse on anomalous and noisy videos, which may have negative consequences when rPPG models are used by medical personnel. To address this, we offer: (a) An anomaly detection model, built on top of the predicted waveforms. We compare models trained in open-set (unknown abnormal predictions) and closed-set (abnormal predictions known when training) settings; (b) An anomaly-aware training regime that penalizes the model for predicting periodic signals from anomalous videos. Extensive experimentation with eight research datasets (rPPG-specific: DDPM, CDDPM, PURE, UBFC, ARPM; deep fakes: DFDC; face presentation attack detection: HKBU-MARs; rPPG outlier: KITTI) show better accuracy of anomaly detection for deep learning models incorporating the proposed training (75.8%), compared to models trained regularly (73.7%) and to hand-crafted rPPG methods (52-62%).

Paper Citation

in Harvard Style

Speth J., Vance N., Sporrer B., Niu L., Flynn P. and Czajka A. (2023). Hallucinated Heartbeats: Anomaly-Aware Remote Pulse Estimation. In Proceedings of the 16th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2023) - Volume 4: BIOSIGNALS; ISBN 978-989-758-631-6, SciTePress, pages 106-117. DOI: 10.5220/0011781700003414

in Bibtex Style

@conference{biosignals23,
author={Jeremy Speth and Nathan Vance and Benjamin Sporrer and Lu Niu and Patrick Flynn and Adam Czajka},
title={Hallucinated Heartbeats: Anomaly-Aware Remote Pulse Estimation},
booktitle={Proceedings of the 16th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2023) - Volume 4: BIOSIGNALS},
year={2023},
pages={106-117},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011781700003414},
isbn={978-989-758-631-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2023) - Volume 4: BIOSIGNALS
TI - Hallucinated Heartbeats: Anomaly-Aware Remote Pulse Estimation
SN - 978-989-758-631-6
AU - Speth J.
AU - Vance N.
AU - Sporrer B.
AU - Niu L.
AU - Flynn P.
AU - Czajka A.
PY - 2023
SP - 106
EP - 117
DO - 10.5220/0011781700003414
PB - SciTePress