×

zbMATH — the first resource for mathematics

Inverse statistical learning. (English) Zbl 1349.62102
Summary: Let \((X,Y)\in\mathcal{X}\times\mathcal{Y}\) be a random couple with unknown distribution \(P\). Let \(\mathcal{G}\) be a class of measurable functions and \(\ell\) a loss function. The problem of statistical learning deals with the estimation of the Bayes: \[ g^{*}=\operatorname{arg} \underset{g\in \mathcal{G}}{\min} \mathbb{E}_{P} \ell(g,(X,Y)). \] In this paper, we study this problem when we deal with a contaminated sample \((Z_{1},Y_{1}),\dots,(Z_{n},Y_{n})\) of i.i.d. indirect observations. Each input \(Z_{i}\), \(i=1,\dots,n\) is distributed from a density \(Af\), where \(A\) is a known compact linear operator and \(f\) is the density of the direct input \(X\).
We derive fast rates of convergence for the excess risk of empirical risk minimizers based on regularization methods, such as deconvolution kernel density estimators or spectral cut-off. These results are comparable to the existing fast rates in [V. Koltchinskii, Ann. Stat. 34, No. 6, 2593–2706 (2006; Zbl 1118.62065)] for the direct case. It gives some insights into the effect of indirect measurements in the presence of fast rates of convergence.

MSC:
62G05 Nonparametric estimation
62H30 Classification and discrimination; cluster analysis (statistical aspects)
PDF BibTeX XML Cite
Full Text: DOI Euclid
References:
[1] Audibert, J.-Y. and Tsybakov, A.B. Fast learning rates for plug-in classifiers. The Annals of Statistics , 35: 608-633, 2007. · Zbl 1118.62041
[2] Bartlett, P.L., Bousquet, O., and Mendelson, S. Local rademacher complexities. The Annals of Statistics , 33(4): 1497-1537, 2005. · Zbl 1083.62034
[3] Bartlett, P.L. and Mendelson, S. Empirical minimization. Probability Theory and Related Fields , 135(3): 311-334, 2006. · Zbl 1142.62348
[4] Blanchard, G., Bousquet, O., and Massart, P. Statistical performance of support vector machines. The Annals of Statistics , 36(2): 489-531, 2008. · Zbl 1133.62044
[5] Bousquet, O. A bennet concentration inequality and its application to suprema of empirical processes. C. R. Acad. SCI. Paris Ser. I Math , 334: 495-500, 2002. · Zbl 1001.60021
[6] Butucea, C. Goodness-of-fit testing and quadratic functionnal estimation from indirect observations. The Annals of Statistics , 35: 1907-1930, 2007. · Zbl 1126.62028
[7] Cavalier, L. Nonparametric statistical inverse problems. Inverse Problems , 24: 1-19, 2008. · Zbl 1137.62323
[8] Delaigle, A., Hall, P., and Meister, A. On deconvolution with repeated measurements. The Annals of Statistics , 36(2): 665-685, 2008. · Zbl 1133.62026
[9] Devroye, L., Györfi, L., and Lugosi, G. A Probabilistic Theory of Pattern Recognition . Springer-Verlag, 1996. · Zbl 0853.68150
[10] Engl, H.W., Hank, M., and Neubauer, A. Regularization of Inverse Problems . Kluwer Academic Publishers Group, Dordrecht, 1996. · Zbl 0859.65054
[11] Fan, J. On the optimal rates of convergence for nonparametric deconvolution problems. The Annals of Statistics , 19: 1257-1272, 1991. · Zbl 0729.62033
[12] Koltchinskii, V. Local rademacher complexities and oracle inequalties in risk minimization. The Annals of Statistics , 34(6): 2593-2656, 2006. · Zbl 1118.62065
[13] Lecué, G. and Mendelson, S. General non-exact oracle inequalities for classes with a subexponential envelope. The Annals of Statistics , 40(2): 832-860, 2012. · Zbl 1274.62247
[14] Lederer, Y. and van de Geer, S. New concentration inequalities for suprema of empirical processes. Submitted, 2012. · Zbl 1355.60026
[15] Loustau, S. Penalized empirical risk minimization over Besov spaces. Electronic Journal of Statistics , 3: 824-850, 2009. · Zbl 1326.62157
[16] Loustau, S. Fast rates for noisy clustering. , 2012.
[17] Loustau, S. and Marteau, C. Discriminant analysis with errors in variables. , 2012. · Zbl 1388.62188
[18] Loustau, S. and Marteau, C. Minimax fast rates in discriminant analysis with errors in variables. In revision to Bernoulli , 2013. · Zbl 1388.62188
[19] Mammen, E. and Tsybakov, A.B. Smooth discrimination analysis. The Annals of Statistics , 27(6): 1808-1829, 1999. · Zbl 0961.62058
[20] Massart, P. Some applications of concentration inequalities to statistics. Ann. Fac. Sci. Toulouse Math. , 9(2): 245-303, 2000. · Zbl 0986.62002
[21] Massart, P. and Nédélec, E. Risk bounds for statistical learning. The Annals of Statistics , 34(5): 2326-2366, 2006. · Zbl 1108.62007
[22] Meister, A. Deconvolution Problems in Nonparametric Statistics . Springer-Verlag, 2009. · Zbl 1178.62028
[23] Polonik, W. Measuring mass concentrations and estimating density contour clusters - An excess mass approach. The Annals of Statistics , 23(3): 855-881, 1995. · Zbl 0841.62045
[24] Talagrand, M. New concentration inequalities in product spaces. Invent. Math , 126: 505-563, 1996. · Zbl 0893.60001
[25] Tsybakov, A.B. Introduction à l’estimation non-paramétrique . Springer-Verlag, 2004a.
[26] Tsybakov, A.B. Optimal aggregation of classifiers in statistical learning. The Annals of Statistics , 32(1): 135-166, 2004b. · Zbl 1105.62353
[27] Tsybakov, A.B. and van de Geer, S. Square root penalty: Adaptation to the margin in classification and in edge estimation. The Annals of Statistics , 33(3): 1203-1224, 2005. · Zbl 1080.62047
[28] van de Geer, S. Empirical Processes in M-estimation . Cambridge University Press, 2000. · Zbl 0953.62049
[29] van der Vaart, A.W. and Wellner, J.A. Weak Convergence and Empirical Processes. With Applications to Statistics . Springer Verlag, 1996. · Zbl 0862.60002
[30] Vapnik, V. Estimation of Dependances Based on Empirical Data . Springer Verlag, 1982. · Zbl 0499.62005
[31] Vapnik, V. The Nature of Statistical Learning Theory . Statistics for Engineering and Information Science, Springer, 2000. · Zbl 0934.62009
[32] Williamson, R.C., Smola, A.J., and Schölkopf, B. Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Transactions on Information Theory , 47(6): 2516-2532, 2001. · Zbl 1008.62507
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.