INCREMENTAL LEARNING OF CONVOLUTIONAL NEURAL NETWORKS

Dušan Medera, Štefan Babinec

Abstract

Convolutional neural networks provide robust feature extraction with ability to learn complex, highdimensional non-linear mappings from collection of examples. To accommodate new, previously unseen data, without the need of retraining the whole network architecture we introduce an algorithm for incremental learning. This algorithm was inspired by AdaBoost algorithm. It utilizes ensemble of modified convolutional neural networks as classifiers by generating multiple hypotheses. Furthermore, with this algorithm we can work with the confidence score of classification, which can play crucial importance in specific real world tasks. This approach was tested on handwritten numbers classification. The classification error achieved by this approach was highly comparable with non-incremental learning.

References

  1. Delakis, M. and Garcia, C. (2003). Training convolutional filters for robust face detection. In Proc. of the IEEE international Workshop of Neural Networks for Signal Processing, pages 739-748.
  2. Freund, Y. and Schapire, R. (1999). A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence, 14:771-780.
  3. Grosberg, S. (1988). Nonlinear neural networks principles, mechanisms and architectures. Neural Networks, 1(1):17-61.
  4. LeCun, Y. (1998). Efficient backprop, neural networks tricks of the trade. Lecture Notes in Computer Science, 1524:9-53.
  5. Polikar, R. (2007). Bootstrap inspired techniques in computational intelligence. IEEE Signal Processing Magazine, 24(4):56-72.
  6. R. Polikar, L. U. and Udpa, S. (2001). Learn++, an incremental learning algorithm for supervised neural networks. IEEE Trans. on Systems, 31(4):497-508.
  7. Valiant, L. (1984). A theory of the learnable. Communications of the ACM, 27:1134-1142.
  8. Y. LeCun, L. Bottou, Y. B. and Haffner, P. (1998). Gradientbased learning applied to document recognition. Proc. of IEEE, 86(11):2278-2324.
Download


Paper Citation


in Harvard Style

Medera D. and Babinec Š. (2009). INCREMENTAL LEARNING OF CONVOLUTIONAL NEURAL NETWORKS . In Proceedings of the International Joint Conference on Computational Intelligence - Volume 1: ICNC, (IJCCI 2009) ISBN 978-989-674-014-6, pages 547-550. DOI: 10.5220/0002316405470550


in Bibtex Style

@conference{icnc09,
author={Dušan Medera and Štefan Babinec},
title={INCREMENTAL LEARNING OF CONVOLUTIONAL NEURAL NETWORKS},
booktitle={Proceedings of the International Joint Conference on Computational Intelligence - Volume 1: ICNC, (IJCCI 2009)},
year={2009},
pages={547-550},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002316405470550},
isbn={978-989-674-014-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Joint Conference on Computational Intelligence - Volume 1: ICNC, (IJCCI 2009)
TI - INCREMENTAL LEARNING OF CONVOLUTIONAL NEURAL NETWORKS
SN - 978-989-674-014-6
AU - Medera D.
AU - Babinec Š.
PY - 2009
SP - 547
EP - 550
DO - 10.5220/0002316405470550