Reducing variance in univariate smoothing. (English) Zbl 1117.62038

Summary: A variance reduction technique in nonparametric smoothing is proposed: at each point of estimation, form a linear combination of a preliminary estimator evaluated at nearby points with the coefficients specified so that the asymptotic bias remains unchanged. The nearby points are chosen to maximize the variance reduction. We study in detail the case of univariate local linear regression. While the new estimator retains many advantages of the local linear estimator, it has appealing asymptotic relative efficiencies. Bandwidth selection rules are available by a simple constant factor adjustment of those for local linear estimation. A simulation study indicates that the finite sample relative efficiency often matches the asymptotic relative efficiency for moderate sample sizes. This technique is very general and has a wide range of applications.


62G08 Nonparametric regression and quantile regression
62G20 Asymptotic properties of nonparametric inference
62G05 Nonparametric estimation
65C60 Computational problems in statistics (MSC2010)


Full Text: DOI arXiv


[1] Abramson, I. S. (1982). On bandwidth variation in kernel estimates—a square root law. Ann. Statist. 10 1217–1223. · Zbl 0507.62040
[2] Brockmann, M., Gasser, T. and Herrmann, E. (1993). Locally adaptive bandwidth choice for kernel regression estimators. J. Amer. Statist. Assoc. 88 1302–1309. JSTOR: · Zbl 0792.62028
[3] Cheng, M.-Y. and Hall, P. (2002). Error-dependent smoothing rules in local linear regression. Statist. Sinica 12 429–447. · Zbl 1011.62042
[4] Cheng, M.-Y. and Hall, P. (2003). Reducing variance in nonparametric surface estimation. J. Multivariate Anal. 86 375–397. · Zbl 1018.62026
[5] Cheng, M.-Y., Hall, P. and Titterington, D. M. (1997). On the shrinkage of local linear curve estimators. Statist. Comput. 7 11–17.
[6] Cheng, M.-Y. and Peng, L. (2006). Simple and efficient improvements of multivariate local linear regression. J. Multivariate Anal. 97 1501–1524. · Zbl 1093.62044
[7] Choi, E. and Hall, P. (1998). On bias reduction in local linear smoothing. Biometrika 85 333–345. JSTOR: · Zbl 0918.62033
[8] Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. J. Amer. Statist. Assoc. 74 829–836. · Zbl 0423.62029
[9] Fan, J. (1993). Local linear regression smoothers and their minimax efficiencies. Ann. Statist. 21 196–216. · Zbl 0773.62029
[10] Fan, J. and Gijbels, I. (1995). Data-driven bandwidth selection in local polynomial fitting: Variable bandwidth and spatial adaptation. J. Roy. Statist. Soc. Ser. B 57 371–394. JSTOR: · Zbl 0813.62033
[11] Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Chapman and Hall, London. · Zbl 0873.62037
[12] Fan, J. and Marron, J. S. (1994). Fast implementations of nonparametric curve estimators. J. Comput. Graph. Statist. 3 35–56.
[13] Gasser, T., Kneip, A. and Köhler, W. (1991). A flexible and fast method for automatic smoothing. J. Amer. Statist. Assoc. 86 643–652. JSTOR: · Zbl 0733.62047
[14] Hall, P. (1992). The Bootstrap and Edgeworth Expansion. Springer, New York. · Zbl 0744.62026
[15] Hall, P. and Turlach, B. A. (1997). Interpolation methods for adapting to sparse design in nonparametric regression (with discussion). J. Amer. Statist. Assoc. 92 466–477. JSTOR: · Zbl 0890.62033
[16] Hastie, T. and Loader, C. (1993). Local regression: Automatic kernel carpentry (with discussion). Statist. Sci. 8 120–143.
[17] Jones, M. C., Linton, O. and Nielsen, J. P. (1995). A simple bias reduction method for density estimation. Biometrika 82 327–338. JSTOR: · Zbl 0823.62033
[18] Kogure, A. (1998). Effective interpolations for kernel density estimators. J. Nonparametr. Statist. 9 165–195. · Zbl 0914.62027
[19] Loader, C. (1999). Local Regression and Likelihood. Springer, New York. · Zbl 0929.62046
[20] Ruppert, D. (1997). Empirical-bias bandwidths for local polynomial nonparametric regression and density estimation. J. Amer. Statist. Assoc. 92 1049–1062. JSTOR: · Zbl 1067.62531
[21] Ruppert, D., Sheather, S. J. and Wand, M. P. (1995). An effective bandwidth selector for local least squares regression. J. Amer. Statist. Assoc. 90 1257–1270. JSTOR: · Zbl 0868.62034
[22] Samiuddin, M. and El-Sayyad, G. M. (1990). On nonparametric kernel density estimates. Biometrika 77 865–874. JSTOR: · Zbl 0712.62033
[23] Seifert, B. and Gasser, T. (1996). Finite-sample variance of local polynomials: Analysis and solutions. J. Amer. Statist. Assoc. 91 267–275. JSTOR: · Zbl 0871.62042
[24] Seifert, B. and Gasser, T. (2000). Data adaptive ridging in local polynomial regression. J. Comput. Graph. Statist. 9 338–360. JSTOR:
[25] Stone, C. J. (1977). Consistent nonparametric regression (with discussion). Ann. Statist. 5 595–645. · Zbl 0366.62051
[26] Tibshirani, R. and Hastie, T. (1987). Local likelihood estimation. J. Amer. Statist. Assoc. 82 559–567. JSTOR: · Zbl 0626.62041
[27] Wand, M. P. and Jones, M. C. (1995). Kernel Smoothing . Chapman and Hall, London. · Zbl 0854.62043
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.