zbMATH — the first resource for mathematics

Examples
Geometry Search for the term Geometry in any field. Queries are case-independent.
Funct* Wildcard queries are specified by * (e.g. functions, functorial, etc.). Otherwise the search is exact.
"Topological group" Phrases (multi-words) should be set in "straight quotation marks".
au: Bourbaki & ti: Algebra Search for author and title. The and-operator & is default and can be omitted.
Chebyshev | Tschebyscheff The or-operator | allows to search for Chebyshev or Tschebyscheff.
"Quasi* map*" py: 1989 The resulting documents have publication year 1989.
so: Eur* J* Mat* Soc* cc: 14 Search for publications in a particular source with a Mathematics Subject Classification code (cc) in 14.
"Partial diff* eq*" ! elliptic The not-operator ! eliminates all results containing the word elliptic.
dt: b & au: Hilbert The document type is set to books; alternatively: j for journal articles, a for book articles.
py: 2000-2015 cc: (94A | 11T) Number ranges are accepted. Terms can be grouped within (parentheses).
la: chinese Find documents in a given language. ISO 639-1 language codes can also be used.

Operators
a & b logic and
a | b logic or
!ab logic not
abc* right wildcard
"ab c" phrase
(ab c) parentheses
Fields
any anywhere an internal document identifier
au author, editor ai internal author identifier
ti title la language
so source ab review, abstract
py publication year rv reviewer
cc MSC code ut uncontrolled term
dt document type (j: journal article; b: book; a: book article)
Least square regression with indefinite kernels and coefficient regularization. (English) Zbl 1225.65015

Let (y i ,x i ) i=1 m be i.i.d. observations with y i , x i X, X being some compact metric space. The authors consider estimates f z for the regression function f ρ (x)=E(y i |x i ), where f z =f α z , f α (x)= i=1 m α i K(x,x i ),

α z =argmin αR m 1 m i=1 m (y i -f α (x i )) 2 +λm i=1 m α i 2 ,

K:X×X is a continuous bounded function (kernel), λ is a regularization parameter.

Consistency of f z is demonstrated under the assumptions that λ=λ(m)0, λ 3/2 m and the true regression function belongs to the closure of {f α } in some suitable reproducing kernel Hilbert space.

To analyze the rates of convergence the authors make assumptions of the form EL -r f ρ (x i ) 2 < for some r>0, where Lf(x)=EK ˜(x,x i )f(x i ), K ˜(x,t)=E u K(x,u)K(t,u). E.g. if r>1 then choosing λ=m 1/5 they get f z -f ρ L 2 =O(m -1/5 ).

Results of simulations are presented for X=[0,1] and the Gaussian kernel K.

MSC:
65C60Computational problems in statistics
62J02General nonlinear regression
46E22Hilbert spaces with reproducing kernels
References:
[1]Aronszajn, N.: Theory of reproducing kernels, Trans. amer. Math. soc. 68, 337-404 (1950) · Zbl 0037.20701 · doi:10.2307/1990404
[2]Bauer, F.; Pereverzev, S.; Rosasco, L.: On regularization algorithms in learning theory, J. complexity 23, 52-72 (2007) · Zbl 1109.68088 · doi:10.1016/j.jco.2006.07.001
[3]Cannon, A.; Ettinger, J. M.; Hush, D.; Scovel, C.: Machine learning with data dependent hypothesis classes, J. Mach learn. Res. 2, 335-358 (2002) · Zbl 1007.68156 · doi:10.1162/153244302760200650
[4]Cucker, F.; Smale, S.: On the mathematical foundations of learning, Bull. amer. Math. soc. 39, 1-49 (2001) · Zbl 0983.68162 · doi:10.1090/S0273-0979-01-00923-5
[5]Cucker, F.; Smale, S.: Best choices for regularization parameters in learning theory: on the bias-variance problem, Found. comput. Math. 2, 413-428 (2002) · Zbl 1057.68085 · doi:10.1007/s102080010030
[6]Evgeniou, T.; Pontil, M.; Poggio, T.: Regularization networks and support vector machines, Adv. comput. Math. 13, 1-50 (2000) · Zbl 0939.68098 · doi:10.1023/A:1018946025316
[7]Lanckriet, G. R. G.; Cristianini, N.; Bartlett, P.; El Ghaoui, L.; Jordan, M. I.: Learning the kernel matrix with semidefinite programming, J. Mach learn. Res. 5, 27-72 (2004) · Zbl 1222.68241 · doi:http://www.jmlr.org/papers/v5/lanckriet04a.html
[8]Liu, C.: Gabor-based kernel PCA with fractional power polynomial models for face recognition, IEEE trans. Pattern anal. Mach intell. 26, 572-581 (2004)
[9]Lo Gerfo, L.; Rosasco, L.; Odone, F.; De Vito, E.; Verri, A.: Spectral algorithms for supervised learning, Neural comput. 20, 1873-1897 (2008) · Zbl 1147.68643 · doi:10.1162/neco.2008.05-07-517
[10]Luss, R.; D’aspremont, A.: Support vector machine classification with indefinite kernels, Math. program. Comput. 1, 97-118 (2009) · Zbl 1191.68511 · doi:10.1007/s12532-009-0005-5
[11]Micchelli, C. A.; Pontil, M.: Learning the kernel function via regularization, J. Mach learn. Res. 6, 1099-1125 (2005) · Zbl 1222.68265 · doi:http://www.jmlr.org/papers/v6/micchelli05a.html
[12]Poggio, T.; Smale, S.: The mathematics of learning: dealing with data, Notices amer. Math. soc. 50, 537-544 (2003) · Zbl 1083.68100 · doi:http://www.ams.org/notices/200305/200305-toc.html
[13]Saigo, H.; Vert, J.; Ueda, N.; Akutsu, T.: Protein homology detection using string alignment kernels, Bioinformatics 20, 1682-1689 (2004)
[14]Scholkopf, B.; Smola, A.: Learning with kernels: support vector machines, regularization, optimization and beyond, (2002)
[15]Smale, S.; Zhou, D. -X.: Shannon sampling and function reconstruction from point values, Bull. amer. Math. soc. (N.S.) 41, 279-305 (2004) · Zbl 1107.94007 · doi:10.1090/S0273-0979-04-01025-0
[16]Smale, S.; Zhou, D. -X.: Shannon sampling. II. connections to learning theory, Appl. comput. Harmon. anal. 19, 285-302 (2005) · Zbl 1107.94008 · doi:10.1016/j.acha.2005.03.001
[17]Smale, S.; Zhou, D. X.: Learning theory estimates via integral operators and their approximations, Constr. approx. 26, 153-172 (2007) · Zbl 1127.68088 · doi:10.1007/s00365-006-0659-y
[18]Sun, H.; Wu, Q.: Regularized least square regression with dependent samples, Adv. comput. Math. 32, No. 2, 175-189 (2010) · Zbl 1191.68535 · doi:10.1007/s10444-008-9099-y
[19]Sun, H.; Wu, Q.: Application of integral operator for regularized least square regression, Math. comput. Modelling 49, 276-285 (2009) · Zbl 1165.45310 · doi:10.1016/j.mcm.2008.08.005
[20]Sun, S.; Wu, Q.: A note on application of integral operator in learning theory, Appl. comput. Harmon. anal. 26, 416-421 (2009) · Zbl 1165.68059 · doi:10.1016/j.acha.2008.10.002
[21]Vapnik, V.: Statistical learning theory, (1998) · Zbl 0935.62007
[22]Wu, Q.; Ying, Y.; Zhou, D. X.: Learning rates of least-square regularized regression, Found. comput. Math. 6, 171-192 (2006) · Zbl 1100.68100 · doi:10.1007/s10208-004-0155-9
[23]Wu, Q.; Ying, Y.; Zhou, D. -X.: Multi-kernel regularized classifiers, J. complexity 23, 108-134 (2007) · Zbl 1171.65043 · doi:10.1016/j.jco.2006.06.007
[24]Wu, Q.; Zhou, D. -X.: SVM soft margin classifiers: linear programming versus quadratic programming, Neural comput. 17, 1160-1187 (2005) · Zbl 1108.90324 · doi:10.1162/0899766053491896
[25]Wu, Q.; Zhou, D. -X.: Learning with sample dependent hypothesis spaces, Comput. math. Appl. 56, 2896-2907 (2008) · Zbl 1165.68388 · doi:10.1016/j.camwa.2008.09.014
[26]Q.W. Xiao, D.X. Zhou, Learning by nonsymmetric kernels with data dependent spaces and 1-regularizer, preprint.
[27]Xu, Y.; Zhang, H.: Refinable kernels, J. Mach learn. Res. 8, 2083-2120 (2007) · Zbl 1222.68337 · doi:http://www.jmlr.org/papers/v8/xu07a.html
[28]Xu, Y.; Zhang, H.: Refinement of reproducing kernels, J. Mach learn. Res. 10, 107-140 (2009)
[29]Zhang, T.: Leave-one-out bounds for kernel methods, Neural comput. 15, 1397-1437 (2003) · Zbl 1085.68144 · doi:10.1162/089976603321780326
[30]Zhang, T.: Learning bounds for kernel regression using effective data dimensionality, Neural comput. 17, 2077-2098 (2005) · Zbl 1080.68044 · doi:10.1162/0899766054323008