Challenges of Generalizing Machine Learning Models in Healthcare

Steven Kessler, Bastian Dewitz, Santhoshkumar Sundarara, Favio Salinas, Artur Lichtenberg, Falko Schmid, Hug Aubin

2025

Abstract

Generalization problems are common in machine learning models, particularly in healthcare applications. This study addresses the issue of real-world generalization and its challenges by analyzing a specific use case: predicting patient readmissions using a Recurrent Neural Network (RNN). Although a previously developed RNN model achieved robust results on the Medical Information Mart for Intensive Care (MIMIC-III) dataset, it showed near-random predictive accuracy when applied to the local hospital’s data (Moazemi et al., 2022). We hypothesize that this discrepancy is due to patient demographics, clinical practices, data collection methods, and healthcare differences in infrastructure. By employing statistical methods and distance metrics for time series, we identified critical disparities in demographic and vital data between the MIMIC and hospital data. These findings highlight possible challenges in developing generalizable machine learning models in healthcare environments and the need to improve not just algorithmic solutions but also the process of measuring and collecting medical data.

Paper Citation

in Harvard Style

Kessler S., Dewitz B., Sundarara S., Salinas F., Lichtenberg A., Schmid F. and Aubin H. (2025). Challenges of Generalizing Machine Learning Models in Healthcare. In Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF; ISBN 978-989-758-731-3, SciTePress, pages 254-260. DOI: 10.5220/0013320200003911

in Bibtex Style

@conference{healthinf25,
author={Steven Kessler and Bastian Dewitz and Santhoshkumar Sundarara and Favio Salinas and Artur Lichtenberg and Falko Schmid and Hug Aubin},
title={Challenges of Generalizing Machine Learning Models in Healthcare},
booktitle={Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF},
year={2025},
pages={254-260},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013320200003911},
isbn={978-989-758-731-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: HEALTHINF
TI - Challenges of Generalizing Machine Learning Models in Healthcare
SN - 978-989-758-731-3
AU - Kessler S.
AU - Dewitz B.
AU - Sundarara S.
AU - Salinas F.
AU - Lichtenberg A.
AU - Schmid F.
AU - Aubin H.
PY - 2025
SP - 254
EP - 260
DO - 10.5220/0013320200003911
PB - SciTePress