×

zbMATH — the first resource for mathematics

Consistency analysis of spectral regularization algorithms. (English) Zbl 1242.68264
Summary: We investigate the consistency of spectral regularization algorithms. We generalize the usual definition of regularization function to enrich the content of spectral regularization algorithms. Under a more general prior condition, using refined error decompositions and techniques of operator norm estimation, satisfactory error bounds and learning rates are proved.

MSC:
68T05 Learning and adaptive systems in artificial intelligence
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] F. Bauer, S. Pereverzev, and L. Rosasco, “On regularization algorithms in learning theory,” Journal of Complexity, vol. 23, no. 1, pp. 52-72, 2007. · Zbl 1109.68088 · doi:10.1016/j.jco.2006.07.001
[2] H. Sun and Q. Wu, “Least square regression with indefinite kernels and coefficient regularization,” Applied and Computational Harmonic Analysis, vol. 30, no. 1, pp. 96-109, 2011. · Zbl 1225.65015 · doi:10.1016/j.acha.2010.04.001
[3] L. Lo Gerfo, L. Rosasco, F. Odone, E. De Vito, and A. Verri, “Spectral algorithms for supervised learning,” Neural Computation, vol. 20, no. 7, pp. 1873-1897, 2008. · Zbl 1147.68643 · doi:10.1162/neco.2008.05-07-517
[4] F. Cucker and S. Smale, “On the mathematical foundations of learning,” Bulletin of the American Mathematical Society, vol. 39, no. 1, pp. 1-49, 2002. · Zbl 0983.68162 · doi:10.1090/S0273-0979-01-00923-5
[5] T. Evgeniou, M. Pontil, and T. Poggio, “Regularization networks and support vector machines,” Advances in Computational Mathematics, vol. 13, no. 1, pp. 1-50, 2000. · Zbl 0939.68098 · doi:10.1023/A:1018946025316
[6] T. Poggio and S. Smale, “The mathematics of learning: dealing with data,” Notices of the American Mathematical Society, vol. 50, no. 5, pp. 537-544, 2003. · Zbl 1083.68100 · www.ams.org
[7] S. Smale and D.-X. Zhou, “Learning theory estimates via integral operators and their approximations,” Constructive Approximation, vol. 26, no. 2, pp. 153-172, 2007. · Zbl 1127.68088 · doi:10.1007/s00365-006-0659-y
[8] H. Sun and Q. Wu, “A note on application of integral operator in learning theory,” Applied and Computational Harmonic Analysis, vol. 26, no. 3, pp. 416-421, 2009. · Zbl 1165.68059 · doi:10.1016/j.acha.2008.10.002
[9] H. Sun and Q. Wu, “Regularized least square regression with dependent samples,” Advances in Computational Mathematics, vol. 32, no. 2, pp. 175-189, 2010. · Zbl 1191.68535 · doi:10.1007/s10444-008-9099-y
[10] Q. Wu, Y. Ying, and D.-X. Zhou, “Learning rates of least-square regularized regression,” Foundations of Computational Mathematics, vol. 6, no. 2, pp. 171-192, 2006. · Zbl 1100.68100 · doi:10.1007/s10208-004-0155-9
[11] T. Zhang, “Leave-one-out bounds for kernel methods,” Neural Computation, vol. 15, pp. 1397-1437, 2003. · Zbl 1085.68144 · doi:10.1162/089976603321780326
[12] V. N. Vapnik, Statistical Learning Theory, A Wiley-Interscience, New York, NY, USA, 1998. · Zbl 0935.62007
[13] Q. Guo and H. W. Sun, “Asymptotic convergence of coefficient regularization based on weakly depended samples,” Journal of Jinan University, vol. 24, no. 1, pp. 99-103, 2010.
[14] Q. Wu and D.-X. Zhou, “Learning with sample dependent hypothesis spaces,” Computers & Mathematics with Applications, vol. 56, no. 11, pp. 2896-2907, 2008. · Zbl 1165.68388 · doi:10.1016/j.camwa.2008.09.014
[15] Y. Yao, L. Rosasco, and A. Caponnetto, “On early stopping in gradient descent learning,” Constructive Approximation, vol. 26, no. 2, pp. 289-315, 2007. · Zbl 1125.62035 · doi:10.1007/s00365-006-0663-2
[16] H. Sun and Q. Wu, “Application of integral operator for regularized least-square regression,” Mathematical and Computer Modelling, vol. 49, no. 1-2, pp. 276-285, 2009. · Zbl 1165.45310 · doi:10.1016/j.mcm.2008.08.005
[17] R. V. Kadison and J. R. Ringrose, Fundamentals of the Theory of Operator Algebras, vol. 1 of Elementary Theory, Academic Press, San Diego, Calif, USA, 1983. · Zbl 0518.46046
[18] I. Pinelis, “Optimum bounds for the distributions of martingales in Banach spaces,” The Annals of Probability, vol. 22, no. 4, pp. 1679-1706, 1994. · Zbl 0836.60015 · doi:10.1214/aop/1176988477
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.