Uniform in bandwidth consistency of kernel-type function estimators. (English) Zbl 1079.62040

Einmahl, Uwe; Mason, David M.

Ann. Stat. 33, No. 3, 1380-1403 (2005).

The authors consider kernel estimators based on iid data in density and regression estimation. Their aim is to prove uniform consistency with rates, where uniformity is meant w.r. to the argument and the smoothing parameter. E.g., let \((X_i)_1^\infty\) be an iid sequence of random vectors following a d-dimensional bounded density \(f\,.\) Under some assumptions on the kernel it is shown that the estimator \(f_{n,h}(x)=(nh)^{-1} \sum_{k=1}^n K((x-X_k)/h^{1/d})\) satisfies with probability one \[ \limsup_{n \to \infty} \sup_{c \log n\,/n\leq h\leq 1}\frac{\sqrt{nh}\| \widehat{f}_{n,h}- E(\widehat{f}_{n,h})\| _\infty}{\sqrt{\log(1/h)\vee \log \log n}}= \kappa(c)<\infty. \] From this result one can, e.g., conclude consistency of estimators with data-driven bandwidths. Related results are shown for the Nadaraya-Watson estimator in regression and for an estimator for conditional distributions. One important tool for the proof is empirical process theory applied to the family of functions \({\mathcal K}=\{K(x-.)/h^{1/d})\), \(x \in {\mathbb R}^d\), \(h>0\}.\)

Reviewer: Ulrich Stadtmüller (Ulm)

MSC:

62G07	Density estimation
62G20	Asymptotic properties of nonparametric inference
62G08	Nonparametric regression and quantile regression
60F15	Strong limit theorems

Keywords:

strong consistency; density estimation; nonparametric regression; uniform consistency w.r.t. bandwidth; empirical processes

References:

[1]	Bosq, D. and Lecoutre, J.-P. (1987). Théorie de l ’ estimation fonctionnelle . Economica, Paris.
[2]	Breiman, L., Meisel, W. and Purcell, E. (1977). Variable kernel estimate of multivariate densities. Technometrics 19 135–144. · Zbl 0379.62023 · doi:10.2307/1268623
[3]	Collomb, G. (1979). Conditions nécessaires et suffisantes de convergence uniforme d’un estimateur de régression, estimation des dérivées de la régression. C. R. Acad. Sci. Paris Sér. A 288 161–163. · Zbl 0397.62042
[4]	Deheuvels, P. (1974). Conditions nécessaires et suffisantes de convergence ponctuelle presque sûre et uniforme presque sûre des estimateurs de la densité. C. R. Acad. Sci. Paris Sér. A 278 1217–1220. · Zbl 0281.62041
[5]	Deheuvels, P. (2000). Uniform limit laws for kernel density estimators on possibly unbounded intervals. In Recent Advances in Reliability Theory : Methodology , Practice and Inference (N. Limnios and M. Nikulin, eds.) 477–492. Birkhäuser, Basel. · Zbl 0997.62038
[6]	Deheuvels, P. and Mason, D. M. (1992). Functional laws of the iterated logarithm for the increments of empirical and quantile processes. Ann. Probab. 20 1248–1287. JSTOR: · Zbl 0760.60028 · doi:10.1214/aop/1176989691
[7]	Deheuvels, P. and Mason, D. M. (2004). General asymptotic confidence bands based on kernel-type function estimators. Stat. Inference Stoch. Process. 7 225–277. · Zbl 1125.62314 · doi:10.1023/B:SISP.0000049092.55534.af
[8]	Devroye, L. and Györfi, L. (1985). Nonparametric Density Estimation : The \(L_1\) View. Wiley, New York. · Zbl 0546.62015
[9]	Devroye, L. and Lugosi, G. (2001). Combinatorial Methods in Density Estimation . Springer, New York. · Zbl 0964.62025
[10]	Eggermont, P. P. B. and LaRiccia, V. N. (2001). Maximum Penalized Likelihood Estimation 1 . Density Estimation . Springer, New York. · Zbl 0984.62026
[11]	Einmahl, U. and Mason, D. M. (2000). An empirical process approach to the uniform consistency of kernel-type function estimators. J. Theoret. Probab. 13 1–37. · Zbl 0995.62042 · doi:10.1023/A:1007769924157
[12]	Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Chapman and Hall, London. · Zbl 0873.62037
[13]	Giné, E. and Guillou, A. (2001). On consistency of kernel density estimators for randomly censored data: Rates holding uniformly over adaptive intervals. Ann. Inst. H. Poincaré Probab. Statist. 37 503–522. · Zbl 0974.62030 · doi:10.1016/S0246-0203(01)01081-0
[14]	Giné, E. and Guillou, A. (2002). Rates of strong uniform consistency for multivariate kernel density estimators. Ann. Inst. H. Poincaré Probab. Statist. 38 907–921. · Zbl 1011.62034 · doi:10.1016/S0246-0203(02)01128-7
[15]	Giné, E. and Zinn, J. (1984). Some limit theorems for empirical processes (with discussion). Ann. Probab. 12 929–998. JSTOR: · Zbl 0553.60037 · doi:10.1214/aop/1176993138
[16]	Härdle, W., Janssen, P. and Serfling, R. (1988). Strong uniform consistency rates for estimators of conditional functionals. Ann. Statist. 16 1428–1449. JSTOR: · Zbl 0672.62050 · doi:10.1214/aos/1176351047
[17]	Hall, P. (1992). On global properties of variable bandwidth density estimators. Ann. Statist. 20 762–778. JSTOR: · Zbl 0785.62040 · doi:10.1214/aos/1176348655
[18]	Jones, M. C., Marron, J. S. and Sheather, S. J. (1996). A brief survey of bandwidth selection for density estimation. J. Amer. Statist. Assoc. 91 401–407. JSTOR: · Zbl 0873.62040 · doi:10.2307/2291420
[19]	Ledoux, M. (1996). On Talagrand’s deviation inequalities for product measures. ESAIM Probab. Statist. 1 63–87. · Zbl 0869.60013 · doi:10.1051/ps:1997103
[20]	Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces. Springer, Berlin. · Zbl 0748.60004
[21]	Lepski, O. V., Mammen, E. and Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogenous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25 929–947. · Zbl 0885.62044 · doi:10.1214/aos/1069362731
[22]	Lepski, O. V. and Spokoiny, V. G. (1997). Optimal pointwise adaptive methods in nonparametric estimation. Ann. Statist. 25 2512–2546. · Zbl 0894.62041 · doi:10.1214/aos/1030741083
[23]	Müller, H.-G. and Stadtmüller, U. (1987). Variable bandwidth kernel estimators of regression curves. Ann. Statist. 15 182–201. JSTOR: · Zbl 0634.62032 · doi:10.1214/aos/1176350260
[24]	Nolan, D. and Marron, J. S. (1989). Uniform consistency of automatic and location-adaptive delta-sequence estimators. Probab. Theory Related Fields 80 619–632. · Zbl 0644.62041 · doi:10.1007/BF00318909
[25]	Nolan, D. and Pollard, D. (1987). \(U\)-processes: Rates of convergence. Ann. Statist. 15 780–799. JSTOR: · Zbl 0624.60048 · doi:10.1214/aos/1176350374
[26]	Pollard, D. (1984). Convergence of Stochastic Processes . Springer, New York. · Zbl 0544.60045
[27]	Staniswalis, J. G. (1989). Local bandwidth selection for kernel estimates. J. Amer. Statist. Assoc. 84 284–288. JSTOR: · Zbl 0676.62039 · doi:10.2307/2289875
[28]	Stein, E. M. (1970). Singular Integrals and Differentiability Properties of Functions . Princeton Univ. Press. · Zbl 0207.13501
[29]	Stute, W. (1982). The oscillation behavior of empirical processes. Ann. Probab . 10 86–107. JSTOR: · Zbl 0489.60038 · doi:10.1214/aop/1176993915
[30]	Stute, W. (1982). The law of the iterated logarithm for kernel density estimators. Ann. Probab . 10 414–422. JSTOR: · Zbl 0493.62040 · doi:10.1214/aop/1176993866
[31]	Stute, W. (1984). The oscillation behavior of empirical processes: The multivariate case. Ann. Probab . 12 361–379. JSTOR: · Zbl 0533.62037 · doi:10.1214/aop/1176993295
[32]	Stute, W. (1986). On almost sure convergence of conditional empirical distribution functions. Ann. Probab. 14 891–901. JSTOR: · Zbl 0593.60043 · doi:10.1214/aop/1176992445
[33]	Talagrand, M. (1994). Sharper bounds for Gaussian and empirical processes. Ann. Probab. 22 28–76. JSTOR: · Zbl 0798.60051 · doi:10.1214/aop/1176988847
[34]	van der Vaart, A. W. (1998). Asymptotic Statistics . Cambridge Univ. Press. · Zbl 0910.62001 · doi:10.1017/CBO9780511802256
[35]	van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes : With Applications to Statistics . Springer, New York. · Zbl 0862.60002