Author:
Przemysław Klęsk
Affiliation:
Westpomeranian University of Technology, Poland
Keyword(s):
Statistical learning theory, Bounds on generalization, Cross-validation, Empirical risk minimization, Structural risk minimization, Vapnik–Chervonenkis dimension.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Computational Intelligence
;
Evolutionary Computing
;
Knowledge Discovery and Information Retrieval
;
Knowledge-Based Systems
;
Machine Learning
;
Soft Computing
;
Symbolic Systems
;
Uncertainty in AI
Abstract:
Typically, the n-fold cross-validation is used both to: (1) estimate the generalization properties of a model of fixed complexity, (2) choose from a family of models of different complexities, the one with the best complexity, given a data set of certain size. Obviously, it is a time-consuming procedure. A different approach — the Structural Risk Minimization is based on generalization bounds of learning machines given by Vapnik (Vapnik, 1995a; Vapnik, 1995b). Roughly speaking, SRM is O(n) times faster than n-fold cross-validation but less accurate.
We state and prove theorems, which show the probabilistic relationship between the two approaches. In particular, we show what e-difference between the two, one may expect without actually performing the crossvalidation. We conclude the paper with results of experiments confronting the probabilistic bounds we derived.