Neural Network Learning: Testing Bounds on Sample Complexity

Joaquim Marques de Sá, Fernando Sereno, Luís Alexandre

Abstract

Several authors have theoretically determined distribution-free bounds on sample complexity. Formulas based on several learning paradigms have been presented. However, little is known on how these formulas perform and compare with each other in practice. To our knowledge, controlled experimental results using these formulas, and comparing of their behavior, have not so far been presented. The present paper represents a contribution to filling up this gap, providing experimentally controlled results on how simple perceptrons trained by gradient descent or by the support vector approach comply with these bounds in practice.

References

  1. Anthony, M., Bartlett, P.L.: Neural Network Learning: Theoretical Foundations. Cambridge University Press (1999)
  2. Baum, E.B., Haussler, D.: What Size Net Gives Valid Generalization? Neural Computation, 1 (1989) 151-160
  3. Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K.: Learnability and the VapnikChernovenkis Dimension. J Ass Comp Machinery, 36 (1989) 929-965
  4. Ehrenfeucht, A., Haussler, D., Kearns, M., Valiant, L.: A General Lower Bound on the Number of Examples Needed for Learning. Information and Computation, 82 (1989) 247-261
  5. Kearns, M.J., Vazirani, U.V.: An Introduction to Computational Learning Theory. The MIT Press (1997)
  6. Gu, H., Takahashi, H.: Towards more practical average bounds on supervised learning. IEEE Trans. Neural Networks, 7 (1996) 953-968
  7. Mitchell, T.M.: Machine Learning. McGraw Hill Book Co. (1997)
  8. Marques-de-Sá, J.P.: Introduction to Statistical Learning Theory. Part I: Data Classification. http://www.fe.up.pt/nnig/ (2003)
  9. Vapnik, V.N.: Statistical Learning Theory. John Wiley & Sons, Inc. (1998)
Download


Paper Citation


in Harvard Style

Marques de Sá J., Sereno F. and Alexandre L. (2004). Neural Network Learning: Testing Bounds on Sample Complexity . In Proceedings of the 4th International Workshop on Pattern Recognition in Information Systems - Volume 1: PRIS, (ICEIS 2004) ISBN 972-8865-01-5, pages 196-201. DOI: 10.5220/0002653301960201


in Bibtex Style

@conference{pris04,
author={Joaquim Marques de Sá and Fernando Sereno and Luís Alexandre},
title={Neural Network Learning: Testing Bounds on Sample Complexity},
booktitle={Proceedings of the 4th International Workshop on Pattern Recognition in Information Systems - Volume 1: PRIS, (ICEIS 2004)},
year={2004},
pages={196-201},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002653301960201},
isbn={972-8865-01-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Workshop on Pattern Recognition in Information Systems - Volume 1: PRIS, (ICEIS 2004)
TI - Neural Network Learning: Testing Bounds on Sample Complexity
SN - 972-8865-01-5
AU - Marques de Sá J.
AU - Sereno F.
AU - Alexandre L.
PY - 2004
SP - 196
EP - 201
DO - 10.5220/0002653301960201