Hierarchically Gated Experts for Efficient Online Continual Learning

Kevin Luong, Michael Thielscher

2025

Abstract

Continual Learning models aim to learn a set of tasks under the constraint that the tasks arrive sequentially with no way to access data from previous tasks. The Online Continual Learning framework poses a further challenge where the tasks are unknown and instead the data arrives as a single stream. Building on existing work, we propose a method for identifying these underlying tasks: the Gated Experts (GE) algorithm, where a dynamically growing set of experts allows for new knowledge to be acquired without catastrophic forgetting. Furthermore, we extend GE to Hierarchically Gated Experts (HGE), a method which is able to efficiently select the best expert for each data sample by organising the experts into a hierarchical structure. On standard Continual Learning benchmarks, GE and HGE are able to achieve results comparable with current methods, with HGE doing so more efficiently.

Paper Citation

in Harvard Style

Luong K. and Thielscher M. (2025). Hierarchically Gated Experts for Efficient Online Continual Learning. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 507-518. DOI: 10.5220/0013190000003890

in Bibtex Style

@conference{icaart25,
author={Kevin Luong and Michael Thielscher},
title={Hierarchically Gated Experts for Efficient Online Continual Learning},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART},
year={2025},
pages={507-518},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013190000003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART
TI - Hierarchically Gated Experts for Efficient Online Continual Learning
SN - 978-989-758-737-5
AU - Luong K.
AU - Thielscher M.
PY - 2025
SP - 507
EP - 518
DO - 10.5220/0013190000003890
PB - SciTePress