×

Quantitative bounds for concentration-of-measure inequalities and empirical regression: the independent case. (English) Zbl 1429.62171

This paper is concerned with the study of the deviation of the average \(L^2\)-error associated to the least squares regressor over a family of functions, obtained from independent but not necessarily identically distributed samples of explanatory and response variables, from the minimal average \(L^2\)-error associated with this family of functions. The authors obtain nonasymptotic deviation inequalities that generalize and refine corresponding results in the independent identically distributed case.

MSC:

62G30 Order statistics; empirical distribution functions
60E15 Inequalities; stochastic orderings

Software:

ElemStatLearn
PDFBibTeX XMLCite
Full Text: DOI HAL

References:

[1] Bennett, G., Probability inequalities for the sum of independent random variables, J. Amer. Statist. Assoc., 57, 297, 33-45 (1962) · Zbl 0104.11905
[2] Bernstein, S. N., On a modification of chebyshev’s inequality and of the error formula of laplace, Ann. Sci. Inst. Sav. Ukraine, Sect. Math., 1 (1924)
[3] Cantelli, F. P., Sulla determinazione empirica delle leggi di probabilità, Giorn. Ist. Ital. Attuari, 4, 421-424 (1933) · JFM 59.1166.05
[4] de la Peña, V. H.; Giné, E., Decoupling. From Dependence to Indepedence. Probability and its Applications (1999), Springer
[5] Dedecker, J.; Doukhan, P.; Lang, G.; León, J. R.; Louhichi, S.; Prieur, C., (Weak Dependence: with Examples and Applications. Weak Dependence: with Examples and Applications, Lecture Notes in Statistics. (2007), Springer) · Zbl 1165.62001
[6] Duffie, D.; Glynn, P. W., Efficient Monte-Carlo simulation of security prices, Ann. Appl. Probab., 5, 4, 897-905 (1995) · Zbl 0877.65099
[7] Fort, G.; Gobet, E.; Moulines, E., Mcmc design-based non-parametric regression for rare-event. application to nested risk computations, Monte Carlo Methods Appl., 23, 1, 21-42 (2017) · Zbl 1360.65024
[8] Giles, M. B., Multilevel Monte-Carlo path simulation, Oper. Res., 56, 607-617 (2008) · Zbl 1167.65316
[9] Giles, M. B., Multilevel Monte Carlo methods, Acta Numer., 24, 259-328 (2015) · Zbl 1316.65010
[10] Glivenko, V., Sulla determinazione empirica delle leggi di probabilità, Giorn. Ist. Ital. Attuari, 4, 92-99 (1933) · JFM 59.1166.04
[11] Györfi, L.; Kohler, M.; Krzyzak, A.; Walk, H., A distribution-free theory of nonparametric regression, (Springer Series in Statistics (2002)) · Zbl 1021.62024
[12] Han, Q.; Jon A. Wellner, A sharp multiplier inequality with applications to heavy-tailed regression problems (2017), arXiv:1706.02410v1
[13] Hastie, T.; Tibshirani, R.; Friedman, J., The elements of statistical learning, (Data Mining, Inference, and Prediction. Data Mining, Inference, and Prediction, Springer Series in Statistics (2009), Springer: Springer New York, NY)
[14] Heinrich, S., Multilevel Monte-Carlo methods, (LSSC ’01 Proceedings of the Third International Conference on Large-Scale Scientific Computing. LSSC ’01 Proceedings of the Third International Conference on Large-Scale Scientific Computing, Lecture Notes in Computer Science, vol. 2179 (2001), Springer-Verlag), 58-67 · Zbl 1031.65005
[15] Hoeffding, W., Probability inequalities for sums of bounded random variables, J. Amer. Statist. Assoc., 58, 301, 13-30 (1963) · Zbl 0127.10602
[16] Kahane, J.-P., Some random series of functions, (Cambridge Studies in Advanced Mathematics (1968), Cambridge University Press: Cambridge University Press Cambridge) · Zbl 0192.53801
[17] Kloeden, P. E.; Platen, E., Numerical solution of stochastic differential equations, (Applications of Mathematics 23 (2010), Springer: Springer Berlin) · Zbl 1216.60052
[18] Kosorok, M. R., Bootstraps of sums of independent but not identically distributed stochastic processes, J. Multivariate Anal., 84, 299-318 (2003) · Zbl 1016.62063
[19] Kuznetzov, V.; Mohri, M., Generalization bounds for non-stationary mixing processes, Mach. Learn., 106, 1, 93-117 (2017) · Zbl 1412.68186
[20] M. Mohri, A. Muñoz Medina, New analysis and algorithm for learning with drifiting distributions. in: Algorithmic Learning Theory. 23rd International Conference, ALT 2012, Lyon, France, October 29-12, 2012. Proceedings. 2012, pp. 29-31.; M. Mohri, A. Muñoz Medina, New analysis and algorithm for learning with drifiting distributions. in: Algorithmic Learning Theory. 23rd International Conference, ALT 2012, Lyon, France, October 29-12, 2012. Proceedings. 2012, pp. 29-31.
[21] Oksendal, B., Stochastic Differential Equations, An Introduction with Applications (2000), Springer-Verlag: Springer-Verlag Heidelberg New York
[22] Pollard, D., Empirical Processes: Theory and Applications (1990), Institute of Mathematical Statistics and American Statistical Association · Zbl 0741.60001
[23] van de Geer, S.; Wegkamp, M., Consistency for the least squares estimator in nonparametric regression, Ann. Statist., 24, 6, 2513-2523 (1996) · Zbl 0867.62027
[24] van der Vaart, A. W.; Wellner, J. A., (Weak Convergence and Empirical Processes with Applications To Statistics. Weak Convergence and Empirical Processes with Applications To Statistics, Springer Series in Statistics (1996), Springer: Springer New York, NY) · Zbl 0862.60002
[25] Vapnik, V. N., The nature of statistical learning theory, (Statistics for Engineering and Information Science (2000), Springer) · Zbl 0934.62009
[26] Vapnik, V. N.; Chervonenkis, Y., On the uniform convergence of relative frequencies of events to their probabilities, Theory Probab. Appl., XVI, 2, 264-280 (1971) · Zbl 0247.60005
[27] Wellner, J. A., A glivenko-cantelli theorem for empirical measures of independent but non-identically distributed random variables, Stochastic Process. Appl., 11, 3, 309-312 (1981) · Zbl 0458.60023
[28] van Zuijlen, M. C.A., Properties of the empirical distribution function for independent nonidentically distributed random variables, Ann. Probab., 6, 2, 250-266 (1978) · Zbl 0396.60040
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.