Development of a Big Data Mechanism for AutoML

Roberto Sá Barreto Paiva da Cunha, Jairson Barbosa Rodrigues, Alexandre M. A. Maciel

2025

Abstract

This paper introduces the development of an AutoML mechanism explicitly designed for large-scale data processing. First, the paper presents a comprehensive technological benchmark of current AutoML frameworks. According to the gaps found, the paper proposes integrating consolidated Big Data technologies into an open-source AutoML framework, emphasizing enhanced usability and scalability in processing capabilities. The entire methodology of this paper was based on Design Science Research - DSR, commonly used in studies that seek to to develop innovative artifacts, such as systems, methods or theoretical models, to address practical challenges. The developed architecture enhances the AutoML FMD - Framework of Data Mining. This integration allowed the efficient management of large datasets and supported distributed machine learning algorithms training. An expert opinion evaluation demonstrated the effectiveness in reducing the learning curve for non-experts and improving scalability and data handling. Integration tests were adopted to validate all FMD components.This work significantly advanced FMD by broadening its applicability to large datasets and various domains while making open-source collaboration and ongoing innovation possible.

Paper Citation

in Harvard Style

Paiva da Cunha R., Rodrigues J. and Maciel A. (2025). Development of a Big Data Mechanism for AutoML. In Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS; ISBN 978-989-758-749-8, SciTePress, pages 294-300. DOI: 10.5220/0013348700003929

in Bibtex Style

@conference{iceis25,
author={Roberto Paiva da Cunha and Jairson Rodrigues and Alexandre Maciel},
title={Development of a Big Data Mechanism for AutoML},
booktitle={Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS},
year={2025},
pages={294-300},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013348700003929},
isbn={978-989-758-749-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS
TI - Development of a Big Data Mechanism for AutoML
SN - 978-989-758-749-8
AU - Paiva da Cunha R.
AU - Rodrigues J.
AU - Maciel A.
PY - 2025
SP - 294
EP - 300
DO - 10.5220/0013348700003929
PB - SciTePress