A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression. (English) Zbl 1303.62035

Chichignoud, Michaël; Lederer, Johannes

Bernoulli 20, No. 3, 1560-1599 (2014).

The authors introduce a robust method for pointwise estimation in heteroscedastic regression. The method is adaptive with respect to the noise and design distribution (D-adaptive) and the smoothness of the regression function (S-adaptive). The noise and design distributions considered are unknown and fulfill weak assumptions – moment conditions on the noise distribution are not imposed and positive density for the design distribution is not required. In a first step, the authors derive general properties of M-estimators such as pointwise risk bounds, including S-minimax results for degenerate designs over unidimensional Hölder spaces. After, they choose the contrast and the kernel of the estimators that minimize an empirical variance term and demonstrate that the corresponding M-estimator is adaptive with respect to the noise and design distributions and adaptive Huber minimax for contamination models. Choosing a data-driven bandwidth via Lepski’s method leads to an M-estimator that is adaptive with respect to the noise and design distributions as well as with respect to the smoothness of an isotropic, multivariate, locally polynomial target function. The results are also extended to anisotropic, locally constant target functions. This approach provides also a level of robustness that adapt to the noise, contamination, and outliers.

Reviewer: Pavel Stoynov (Sofia)

MSC:

62G35	Nonparametric robustness
62G08	Nonparametric regression and quantile regression
62G20	Asymptotic properties of nonparametric inference
62C20	Minimax procedures in statistical decision theory

Keywords:

adaptation; Huber contrast; Lepski’s method; M-estimation; minimax estimation; nonparametric regression; pointwise estimation; robust estimation

References:

[1]	Antoniadis, A., Pensky, M. and Sapatinas, T. (2014). Nonparametric regression estimation based on spatially inhomogeneous data: Minimax global convergence rates and adaptivity. ESAIM Probab. Stat. 18 1-41. · Zbl 1305.62167 · doi:10.1051/ps/2012024
[2]	Arcones, M.A. (2005). Convergence of the optimal \(M\)-estimator over a parametric family of \(M\)-estimators. Test 14 281-315. · Zbl 1069.62018 · doi:10.1007/BF02595407
[3]	Bertin, K. (2004). Estimation asymptotiquement exacte en norme sup de fonctions multidimensionnelles. Ph.D. thesis, Paris 6.
[4]	Brown, L.D. and Low, M.G. (1996). A constrained risk inequality with applications to nonparametric functional estimation. Ann. Statist. 24 2524-2535. · Zbl 0867.62023 · doi:10.1214/aos/1032181166
[5]	Cai, T.T. and Zhou, H.H. (2009). Asymptotic equivalence and adaptive estimation for robust nonparametric regression. Ann. Statist. 37 3204-3235. · Zbl 1191.62070 · doi:10.1214/08-AOS681
[6]	Chichignoud, M. (2012). Minimax and minimax adaptive estimation in multiplicative regression: Locally Bayesian approach. Probab. Theory Related Fields 153 543-586. · Zbl 1318.62128 · doi:10.1007/s00440-011-0354-7
[7]	Gaïffas, S. (2005). Convergence rates for pointwise curve estimation with a degenerate design. Math. Methods Statist. 14 1-27.
[8]	Gaïffas, S. (2007). On pointwise adaptive curve estimation based on inhomogeneous data. ESAIM Probab. Stat. 11 344-364 (electronic). · Zbl 1187.62074 · doi:10.1051/ps:2007023
[9]	Gaïffas, S. (2007). Sharp estimation in sup norm with random design. Statist. Probab. Lett. 77 782-794. · Zbl 1114.62046 · doi:10.1016/j.spl.2006.11.017
[10]	Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122-1170. · Zbl 1183.62062 · doi:10.1214/09-AOS738
[11]	Goldenshluger, A. and Lepski, O. (2008). Universal pointwise selection rule in multivariate function estimation. Bernoulli 14 1150-1190. · Zbl 1168.62323 · doi:10.3150/08-BEJ144
[12]	Goldenshluger, A. and Nemirovski, A. (1997). On spatially adaptive estimation of nonparametric regression. Math. Methods Statist. 6 135-170. · Zbl 0892.62018
[13]	Hoffmann, M. and Nickl, R. (2011). On adaptive inference and confidence bands. Ann. Statist. 39 2383-2409. · Zbl 1232.62072 · doi:10.1214/11-AOS903
[14]	Huber, P.J. (1964). Robust estimation of a location parameter. Ann. Math. Statist. 35 73-101. · Zbl 0136.39805 · doi:10.1214/aoms/1177703732
[15]	Huber, P.J. (1981). Robust Statistics. Wiley Series in Probability and Mathematical Statistics . New York: Wiley. · Zbl 0536.62025
[16]	Huber, P.J. and Ronchetti, E.M. (2009). Robust Statistics , 2nd ed. Wiley Series in Probability and Statistics . Hoboken, NJ: Wiley. · Zbl 1276.62022
[17]	Katkovnik, V., Foi, A., Egiazarian, K. and Astola, J. (2010). From local kernel to nonlocal multiple-model image denoising. Int. J. Comput. Vis. 86 1-32. · Zbl 1477.94014 · doi:10.1007/s11263-009-0272-7
[18]	Katkovnik, V.Y. (1985). Neparametricheskaya Identifikatsiya i Sglazhivanie Dannykh. Metod Lokalnoi Approksimatsii . [ The Method of Local Approximation ]. Teoreticheskie Osnovy Tekhnicheskoĭ Kibernetiki . [ Theoretical Foundations of Engineering Cybernetics ]. Moscow: “Nauka”. · Zbl 0576.62050
[19]	Kerkyacharian, G., Lepski, O. and Picard, D. (2001). Nonlinear estimation in anisotropic multi-index denoising. Probab. Theory Related Fields 121 137-170. · Zbl 1010.62029 · doi:10.1007/s004400100148
[20]	Klutchnikoff, N. (2005). Sur l’estimation adaptative de fonctions anisotropes. Ph.D. thesis, Aix-Marseille 1.
[21]	Lambert-Lacroix, S. and Zwald, L. (2011). Robust regression through the Huber’s criterion and adaptive lasso penalty. Electron. J. Stat. 5 1015-1053. · Zbl 1274.62467 · doi:10.1214/11-EJS635
[22]	Lepski, O.V. and Levit, B.Y. (1999). Adaptive nonparametric estimation of smooth multivariate functions. Math. Methods Statist. 8 344-370. · Zbl 1104.62312
[23]	Lepski, O.V., Mammen, E. and Spokoiny, V.G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25 929-947. · Zbl 0885.62044 · doi:10.1214/aos/1069362731
[24]	Lepskiĭ, O.V. (1990). A problem of adaptive estimation in Gaussian white noise. Teor. Veroyatn. Primen. 35 459-470. · Zbl 0725.62075
[25]	Massart, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Math. 1896 . Berlin: Springer. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6-23, 2003, with a foreword by Jean Picard.
[26]	Polzehl, J. and Spokoiny, V. (2006). Propagation-separation approach for local likelihood estimation. Probab. Theory Related Fields 135 335-362. · Zbl 1089.62033 · doi:10.1007/s00440-005-0464-1
[27]	Reiss, M., Rozenholc, Y. and Cuenod, C. (2009). Pointwise adaptive estimation for robust and quantile regression. Available at . 0904.0543v1
[28]	Spokoiny, V. and Vial, C. (2009). Parameter tuning in pointwise adaptation using a propagation approach. Ann. Statist. 37 2783-2807. · Zbl 1173.62028 · doi:10.1214/08-AOS607
[29]	Stone, C.J. (1975). Adaptive maximum likelihood estimators of a location parameter. Ann. Statist. 3 267-284. · Zbl 0303.62026 · doi:10.1214/aos/1176343056
[30]	Tsybakov, A.B. (1982). Nonparametric signal estimation when there is incomplete information on the noise distribution. Probl. Inf. Transm. 18 44-60. · Zbl 0499.94003
[31]	Tsybakov, A.B. (1982). Robust estimates of a function. Problemy Peredachi Informatsii 18 39-52. · Zbl 0499.62027
[32]	Tsybakov, A.B. (1983). Convergence of nonparametric robust algorithms of reconstruction of functions. Avtomat. i Telemekh. 12 66-76. · Zbl 0539.62047
[33]	Tsybakov, A.B. (1986). Robust reconstruction of functions by a local approximation method. Problemy Peredachi Informatsii 22 69-84. · Zbl 0622.62047
[34]	Tsybakov, A.B. (2009). Introduction to Nonparametric Estimation. Springer Series in Statistics . New York: Springer. Revised and extended from the 2004 French original, translated by Vladimir Zaiats. · Zbl 1176.62032
[35]	van de Geer, S. and Lederer, J. (2013). The Bernstein-Orlicz norm and deviation inequalities. Probab. Theory Related Fields 157 225-250. · Zbl 1284.60060 · doi:10.1007/s00440-012-0455-y