Model selection for regularized least-squares algorithm in learning theory. (English) Zbl 1083.68106

Vito, E.; Caponnetto, A.; Rosasco, L.

Found. Comput. Math. 5, No. 1, 59-85 (2005).

Summary: We investigate the problem of model selection for learning algorithms depending on a continuous parameter. We propose a model selection procedure based on a worst-case analysis and on a data-independent choice of the parameter. For the regularized least-squares algorithm we bound the generalization error of the solution by a quantity depending on a few known constants and we show that the corresponding model selection procedure reduces to solving a bias-variance problem. Under suitable smoothness conditions on the regression function, we estimate the optimal parameter as a function of the number of data and we prove that this choice ensures consistency of the algorithm.

MSC:

68T05	Learning and adaptive systems in artificial intelligence
68P30	Coding and information theory (compaction, compression, models of communication, encoding schemes, etc.) (aspects in computer science)

Keywords:

learning algorithms