Multiple Additive Neural Networks: A Novel Approach to Continuous Learning in Regression and Classification

Janis Mohr, Basile Tousside, Marco Schmidt, Jörg Frochte

2023

Abstract

Gradient Boosting is one of the leading techniques for the regression and classification of structured data. Recent adaptations and implementations use decision trees as base learners. In this work, a new method based on the original approach of Gradient Boosting was adapted to nearly shallow neural networks as base learners. The proposed method supports a new architecture-based approach for continuous learning and utilises strong heuristics against overfitting. Therefore, the method that we call Multiple Additive Neural Networks (MANN) is robust and achieves high accuracy. As shown by our experiments, MANN obtains more accurate predictions on well-known datasets than Extreme Gradient Boosting (XGB), while also being less prone to overfitting and less dependent on the selection of the hyperparameters learn rate and iterations.

Paper Citation

in Harvard Style

Mohr J., Tousside B., Schmidt M. and Frochte J. (2023). Multiple Additive Neural Networks: A Novel Approach to Continuous Learning in Regression and Classification. In Proceedings of the 15th International Joint Conference on Computational Intelligence - Volume 1: NCTA; ISBN 978-989-758-674-3, SciTePress, pages 540-547. DOI: 10.5220/0012234000003595

in Bibtex Style

@conference{ncta23,
author={Janis Mohr and Basile Tousside and Marco Schmidt and Jörg Frochte},
title={Multiple Additive Neural Networks: A Novel Approach to Continuous Learning in Regression and Classification},
booktitle={Proceedings of the 15th International Joint Conference on Computational Intelligence - Volume 1: NCTA},
year={2023},
pages={540-547},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012234000003595},
isbn={978-989-758-674-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Joint Conference on Computational Intelligence - Volume 1: NCTA
TI - Multiple Additive Neural Networks: A Novel Approach to Continuous Learning in Regression and Classification
SN - 978-989-758-674-3
AU - Mohr J.
AU - Tousside B.
AU - Schmidt M.
AU - Frochte J.
PY - 2023
SP - 540
EP - 547
DO - 10.5220/0012234000003595
PB - SciTePress