Application of integral operator for regularized least-square regression. (English) Zbl 1165.45310

Summary: We study the consistency of the regularized least-square regression in a general reproducing kernel Hilbert space. We characterize the compactness of the inclusion map from a reproducing kernel Hilbert space to the space of continuous functions and show that the capacity-based analysis by uniform covering numbers may fail in a very general setting. We prove the consistency and compute the learning rate by means of integral operator techniques. To this end, we study the properties of the integral operator. The analysis reveals that the essence of this approach is the isomorphism of the square root operator.


45P05 Integral operators
Full Text: DOI


[1] Alon, N.; Ben-David, S.; Cesa-Bianchi, N.; Haussler, D., Scale-sensitive dimension, uniform convergence and learnability, J. Assoc. Comput. Mach., 44, 615-631 (1997) · Zbl 0891.68086
[2] Aronszajn, N., Theory of reproducing kernels, Trans. Amer. Math. Soc., 68, 337-404 (1950) · Zbl 0037.20701
[3] Bartlett, P. L.; Mendelson, S., Rademacher and Gaussian complexities: Risk bounds and structural results, J. Mach. Learn. Res., 3, 463-482 (2002) · Zbl 1084.68549
[4] Cucker, F.; Smale, S., On the mathematical foundations of learning, Bull. Amer. Soc., 39, 1-49 (2001) · Zbl 0983.68162
[5] Cucker, F.; Smale, S., Best choices for regularization parameters in learning theory, Found. Comput. Math., 2, 413-428 (2002) · Zbl 1057.68085
[6] Douglas, R. G., Banach Algebra Techniques in Operator Theory (1998), Springer · Zbl 0920.47001
[7] Hirsch, F.; Lacombe, G., Elements of Functional Analysis (1999), Springer-Verlag: Springer-Verlag New York · Zbl 0924.46001
[8] Poggio, T.; Smale, S., The mathematics of learning: Dealing with data, Notices Amer. Math. Soc., 50, 537-544 (2003) · Zbl 1083.68100
[9] Kadison, Richard V.; Ringrose, John R., Fundamentals of the Theory of Operator Algebras, Volume 1: Elementary Theory (1983), Academic Press: Academic Press San Diego, California · Zbl 0518.46046
[10] Smale, S.; Zhou, D. X., Shannon sampling II. Connections to learning theory, Appl. Comput. Harmon. Anal., 19, 285-302 (2006) · Zbl 1107.94008
[11] Smale, S.; Zhou, D. X., Learning theory estimates via integral operators and their approximations, Constr. Approx., 26, 153-172 (2007) · Zbl 1127.68088
[12] Smale, S.; Zhou, D. X., Shannon sampling and function reconstruction from point values, Bull. Amer. Math. Soc., 41, 279-305 (2004) · Zbl 1107.94007
[13] Sun, H. W., Mercer theorem for RKHS on noncompact sets, J. Complexity, 21, 337-349 (2005) · Zbl 1094.46021
[14] H.W. Sun, Q. Wu, Regularized least square regression with dependent samples, Adv. Comput. Math., in press (doi:10.1007/s10444-008-9099-y); H.W. Sun, Q. Wu, Regularized least square regression with dependent samples, Adv. Comput. Math., in press (doi:10.1007/s10444-008-9099-y) · Zbl 1191.68535
[15] Sun, H. W.; Zhou, D. X., Reproducing kernel Hilbert spaces associated with analytic translation-invariant Mercer kernels, J. Fourier Anal. Appl., 14, 89-101 (2008) · Zbl 1153.46017
[16] Vapnik, V., Statistical Learning Theory (1998), Wiley: Wiley New York · Zbl 0935.62007
[17] Wu, Q.; Ying, Y. M.; Zhou, D. X., Learning rates of least-square regularized regression, Found. Comput. Math., 6, 171-192 (2006) · Zbl 1100.68100
[18] Zhou, D. X., Capacity of reproducing kernel spaces in learning theory, IEEE Trans. Inform. Theory, 49, 1743-1752 (2003) · Zbl 1290.62033
[19] Zhou, D. X., The covering number in learning theory, J. Complexity, 18, 739-767 (2002) · Zbl 1016.68044
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.