×

Integral operator approach to learning theory with unbounded sampling. (English) Zbl 1285.68143

Summary: This paper mainly focuses on the least square regularized regression learning algorithm in a setting of unbounded sampling. Our task is to establish learning rates by means of integral operators. By imposing a moment hypothesis on the unbounded sampling outputs and a function space condition associated with marginal distribution \(\rho_X\), we derive learning rates which are consistent with those in the bounded sampling setting.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
62J02 General nonlinear regression
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Bennett C., Sharpley R.: Interpolation of Operators. Academic Press, Boston (1988) · Zbl 0647.46057
[2] Bennett G.: Probability inequalities for the sum of indpendent random variables. J. Am. Stat. Assoc. 57, 33–45 (1962) · Zbl 0104.11905 · doi:10.1080/01621459.1962.10482149
[3] Caponnetto A., De Vito E.: Optimal rates for regularized least squares algorithm. Found. Comput. Math. 7, 331–368 (2007) · Zbl 1129.68058 · doi:10.1007/s10208-006-0196-8
[4] Cucker F., Smale S.: On the mathmatical foundations of learning. Am. Math. Soc. 39, 1–49 (2001) · Zbl 0983.68162 · doi:10.1090/S0273-0979-01-00923-5
[5] Cucker F., Zhou D.X.: Learning Theory: An Approximation Theory Viewpoint. Cambridge University Press, Cambridge (2007) · Zbl 1274.41001
[6] Evgeniou T., Pontil M., Poggio T.: Reguarization networks and support vector machines. Adv. Comput. Math. 13, 1–50 (2000) · Zbl 0939.68098 · doi:10.1023/A:1018946025316
[7] Guo, Z.C., Zhou, D.X.: Concentration estimates for learning with unbounded sampling (preprint, 2010)
[8] Mendelson S., Neeman J.: Regularization in kernel learning. Ann. Stat. 38, 526–565 (2010) · Zbl 1191.68356 · doi:10.1214/09-AOS728
[9] Pinelis I.F., Sakhanenko A.I.: Remarks on inequalities for probabilities of large deviations. Theory Probab. Appl. 30, 143–148 (1985) · Zbl 0583.60023 · doi:10.1137/1130013
[10] Smale S., Zhou D.X.: Shannon sampling II. Connection to learning theory. Appl. Comput. Harmonic Anal. 19, 285–312 (2005) · Zbl 1107.94008 · doi:10.1016/j.acha.2005.03.001
[11] Smale S., Zhou D.X.: Learning Theory estimates via intergal operators and their approximations. Constr. Approx. 26, 153–172 (2007) · Zbl 1127.68088 · doi:10.1007/s00365-006-0659-y
[12] Smale S., Zhou D.X.: Geometry on probability spaces. Constr. Approx. 30, 311–323 (2009) · Zbl 1187.68270 · doi:10.1007/s00365-009-9070-2
[13] Smale S., Zhou D.X.: Online learning with Markov sampling. Anal. Appl. 7, 87–113 (2009) · Zbl 1170.68022 · doi:10.1142/S0219530509001293
[14] Steinwart, I., Hush, D., Scovel, C.: Optimal rates for regularized least squares regression. In: Dasgupta, S., Klivans, A. (eds.) Proceedings of the 22nd Annual Conference on Learning Theory, pp. 79–93 (2009)
[15] Sun H.W., Wu Q.: Regularized least equare regression with dependent samples. Adv. Comput. Math. 11, 235–249 (2008)
[16] Sun H.W., Wu Q.: A note on application of integral operator in learning theory. Appl. Comput. Harmonic Anal. 26, 416–421 (2009) · Zbl 1165.68059 · doi:10.1016/j.acha.2008.10.002
[17] Wang C., Zhou D.X.: Optimal learning rates for least square regularized regression with unbounded sampling. J. Complexity 27, 55–67 (2011) · Zbl 1217.65024 · doi:10.1016/j.jco.2010.10.002
[18] Wu Q., Ying Y.M., Zhou D.X.: Learning rates of least-square regularized regression. Found. Comput. Math. 6, 171–192 (2006) · Zbl 1100.68100 · doi:10.1007/s10208-004-0155-9
[19] Ye G.B., Zhou D.X.: SVM learning and L p approximation by Gaussians on Riemannian manifolds. Anal. Appl. 7, 309–339 (2009) · Zbl 1175.68346 · doi:10.1142/S0219530509001384
[20] Yurinsky Y.: Sums and Gaussian Vectors. Lecture Notes in Mathematics. Springer, Berlin, Heidelberg (1995) · Zbl 0846.60003
[21] Zhang T.: Leave-one-out bounds for kernel methods. Neural Comput. 15, 1397–1437 (2003) · Zbl 1085.68144 · doi:10.1162/089976603321780326
[22] Zhou D.X.: Capacity of reproducing kernel spaces in learning theory. IEEE Trans. Inform. Theory 49, 1743–1752 (2003) · Zbl 1290.62033 · doi:10.1109/TIT.2003.813564
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.