A Survey of Evaluating AutoML and Automated Feature Engineering Tools in Modern Data Science

Dinesha Dissanayake, Rajitha Navarathna, Praveen Ekanayake, Sumanaruban Rajadurai

2025

Abstract

This survey provides a comprehensive comparison of several AutoML tools, along with an evaluation of three feature engineering tools: Featuretools, Autofeat, and PyCaret. We conducted a benchmarking analysis of four AutoML tools (TPOT, H2O-AutoML, PyCaret, and AutoGluon) using seven datasets sourced from OpenML and the UCI Machine Learning Repository, covering binary classification, multiclass classification, and regression tasks. Key metrics such as F1-score for classification and RMSE for regression were used to assess performance. The tools are also compared in terms of execution time, memory usage, and optimization success. AutoGluon consistently demonstrated strong predictive performance, while H2O-AutoML showed reliable results but was limited by long optimization times. PyCaret was the most efficient, showing notably shorter execution times and lower memory usage across all datasets compared to other tools, though it had slightly lower accuracy. TPOT frequently struggled to complete optimization within the set time limit, achieving successful completion in only 42.86% of total cases. Overall, this survey provides insights into which AutoML tools are best suited for different task requirements.

Paper Citation

in Harvard Style

Dissanayake D., Navarathna R., Ekanayake P. and Rajadurai S. (2025). A Survey of Evaluating AutoML and Automated Feature Engineering Tools in Modern Data Science. In Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS; ISBN 978-989-758-749-8, SciTePress, pages 218-225. DOI: 10.5220/0013266700003929

in Bibtex Style

@conference{iceis25,
author={Dinesha Dissanayake and Rajitha Navarathna and Praveen Ekanayake and Sumanaruban Rajadurai},
title={A Survey of Evaluating AutoML and Automated Feature Engineering Tools in Modern Data Science},
booktitle={Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS},
year={2025},
pages={218-225},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013266700003929},
isbn={978-989-758-749-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS
TI - A Survey of Evaluating AutoML and Automated Feature Engineering Tools in Modern Data Science
SN - 978-989-758-749-8
AU - Dissanayake D.
AU - Navarathna R.
AU - Ekanayake P.
AU - Rajadurai S.
PY - 2025
SP - 218
EP - 225
DO - 10.5220/0013266700003929
PB - SciTePress