Mitigate Catastrophic Forgetting by Varying Goals

Lu Chen, Murata Masayuki

2020

Abstract

Catastrophic forgetting occurs because neural network learning algorithms change connections to learn a new skill which encodes previously acquired skills. Recent research suggests that encouraging modularity in neural networks may overcome catastrophic forgetting because it should reduce learning interference. However, manually constructing modular topology is hard in practice since it involves expert design and trial and error. Therefore, an automatic approach is needed. Kashtan et al. find that evolution under an environment that changes in a modular fashion can lead to the spontaneous evolution of modular network structure. However, goals in their research are made of a different combination of subgoals, while real-world data is rarely perfectly separable. Therefore, in this paper, we explore the application of such approach to mitigate catastrophic forgetting in a slightly practical situation, that is applying it to classification of small sized real images and applying it to the increment of goals. We find that varying goals can improve catastrophic forgetting in a CIFAR-10 based classification problem. We find that when learning a large set of goals, a relatively small switching interval is required to have the advantage of mitigating catastrophic forgetting. On the other hand, when learning a small set of goals, an appropriate large switching interval is preferred since this less worsens the advantage and also can improve accuracy.

Paper Citation

in Harvard Style

Chen L. and Masayuki M. (2020). Mitigate Catastrophic Forgetting by Varying Goals. In Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-395-7, pages 530-537. DOI: 10.5220/0008950005300537

in Bibtex Style

@conference{icaart20,
author={Lu Chen and Murata Masayuki},
title={Mitigate Catastrophic Forgetting by Varying Goals},
booktitle={Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2020},
pages={530-537},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008950005300537},
isbn={978-989-758-395-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Mitigate Catastrophic Forgetting by Varying Goals
SN - 978-989-758-395-7
AU - Chen L.
AU - Masayuki M.
PY - 2020
SP - 530
EP - 537
DO - 10.5220/0008950005300537