INVERSE PROBLEMS IN LEARNING FROM DATA

Věra Kůrková

Abstract

It is shown that application of methods from theory of inverse problems to learning from data leads to simple proofs of characterization of minima of empirical and expected error functionals and their regularized versions. The reformulation of learning in terms of inverse problems also enables comparison of regularized and non regularized case showing that regularization achieves stability by merely modifying output weights of global minima. Methods of theory of inverse problems lead to choice of reproducing kernel Hilbert spaces as suitable ambient function spaces.

References

  1. Aronszajn, N. (1950). Theory of reproducing kernels. Transactions of AMS, 68:337-404.
  2. Bertero, M. (1989). Linear inverse and ill-posed problems. Advances in Electronics and Electron Physics, 75:1- 120.
  3. Bishop, C. (1995). Training with noise is equivalent to Tikhonov regularization. Neural Computation, 7(1):108-116.
  4. Cucker, F. and Smale, S. (2002). On the mathematical foundations of learning. Bulletin of AMS, 39:1-49.
  5. Engl, E. W., Hanke, M., and Neubauer, A. (1999). Regularization of Inverse Problems. Kluwer, Dordrecht.
  6. Fine, T. L. (1999). Feedforward Neural Network Methodology. Springer-Verlag, Berlin, Heidelberg.
  7. Friedman, A. (1982). Modern Analysis. Dover, New York.
  8. Girosi, F., Jones, M., and Poggio, T. (1995). Regularization theory and neural networks architectures. Neural Computation, 7:219-269.
  9. Girosi, F. and Poggio, T. (1990). Regularization algorithms for learning that are equivalent to multilayer networks. Science, 247(4945):978-982.
  10. Groetch, C. W. (1977). Generalized Inverses of Linear Operators. Dekker, New York.
  11. Hansen, P. C. (1998). Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia.
  12. Ito, Y. (1992). Finite mapping by neural networks and truth functions. Mathematical Scientist, 17:69-77.
  13. Kecman, V. (2001). Learning and Soft Computing. MIT Press, Cambridge.
  14. Ku°rková, V. and Sanguineti, M. (2005a). Error estimates for approximate optimization by the extended Ritz method. SIAM Journal on Optimization, 15:461-487.
  15. Ku°rková, V. and Sanguineti, M. (2005b). Learning with generalization capability by kernel methods with bounded complexity. Journal of Complexity, 13:551- 559.
  16. Michelli, C. A. (1986). Interpolation of scattered data: Distance matrices and conditionally positive definite functions. Constructive Approximation, 2:11-22.
  17. Moore, E. H. (1920). Abstract. Bulletin of AMS, 26:394- 395.
  18. Penrose, R. (1955). A generalized inverse for matrices. Proceedings of Cambridge Philosophical Society, 51:406-413.
  19. Poggio, T. and Smale, S. (2003). The mathematics of learning: dealing with data. Notices of AMS, 50:537-544.
  20. Schölkopf, B. and Smola, A. J. (2002). Learning with Kernels - Support Vector Machines, Regularization, Optimization and Beyond. MIT Press, Cambridge.
  21. Tikhonov, A. N. and Arsenin, V. Y. (1977). Solutions of Ill-posed Problems. W.H. Winston, Washington, D.C.
  22. Vito, E. D., Rosasco, L., Caponnetto, A., Giovannini, U. D., and Odone, F. (2005). Learning from examples as an inverse problem. Journal of Machine Learning Research, 6:883-904.
  23. Wahba, G. (1990). Splines Models for Observational Data. SIAM, Philadelphia.
Download


Paper Citation


in Harvard Style

Kůrková V. (2010). INVERSE PROBLEMS IN LEARNING FROM DATA . In Proceedings of the International Conference on Fuzzy Computation and 2nd International Conference on Neural Computation - Volume 1: ICNC, (IJCCI 2010) ISBN 978-989-8425-32-4, pages 316-321. DOI: 10.5220/0003079003160321


in Bibtex Style

@conference{icnc10,
author={Věra Kůrková},
title={INVERSE PROBLEMS IN LEARNING FROM DATA},
booktitle={Proceedings of the International Conference on Fuzzy Computation and 2nd International Conference on Neural Computation - Volume 1: ICNC, (IJCCI 2010)},
year={2010},
pages={316-321},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003079003160321},
isbn={978-989-8425-32-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Fuzzy Computation and 2nd International Conference on Neural Computation - Volume 1: ICNC, (IJCCI 2010)
TI - INVERSE PROBLEMS IN LEARNING FROM DATA
SN - 978-989-8425-32-4
AU - Kůrková V.
PY - 2010
SP - 316
EP - 321
DO - 10.5220/0003079003160321