Benchmarking Auto-WEKA on a Commodity Machine

João Freitas, Nuno Lavado, Jorge Bernardino

2018

Abstract

Machine Learning model building is an important and complex task in Data Science but also a good target for automation as recently exploited by AutoML. In general, free and open-source packages offer a joint space of learning algorithms and their respective hyperparameter settings and an optimization method for model search and tuning. In this paper, Auto-WEKA’s performance has been tested by running it for short periods of time (5, 15 and 30 minutes) using a commodity machine and suitable datasets with a limited number of observations and features. Benchmarking was performed against the best human-generated solution available in OpenML for each selected dataset. We concluded that increasing the overall time budget available over the previous values didn’t improve significantly classifiers’ performance.

Paper Citation

in Harvard Style

Freitas J., Lavado N. and Bernardino J. (2018). Benchmarking Auto-WEKA on a Commodity Machine.In Proceedings of the 7th International Conference on Data Science, Technology and Applications - Volume 1: DATA, ISBN 978-989-758-318-6, pages 180-186. DOI: 10.5220/0006914801800186

in Bibtex Style

@conference{data18,
author={João Freitas and Nuno Lavado and Jorge Bernardino},
title={Benchmarking Auto-WEKA on a Commodity Machine},
booktitle={Proceedings of the 7th International Conference on Data Science, Technology and Applications - Volume 1: DATA,},
year={2018},
pages={180-186},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006914801800186},
isbn={978-989-758-318-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 7th International Conference on Data Science, Technology and Applications - Volume 1: DATA,
TI - Benchmarking Auto-WEKA on a Commodity Machine
SN - 978-989-758-318-6
AU - Freitas J.
AU - Lavado N.
AU - Bernardino J.
PY - 2018
SP - 180
EP - 186
DO - 10.5220/0006914801800186