# zbMATH — the first resource for mathematics

Kernel smoothers: an overview of curve estimators for the first graduate course in nonparametric statistics. (English) Zbl 1127.62330
Summary: An introduction to nonparametric regression is accomplished with selected real data sets, statistical graphics and simulations from known functions. It is pedagogically effective for many to have some initial intuition about what the techniques are and why they work. Visual displays of small examples along with the plots of several types of smoothers are a good beginning. Some students benefit from a brief historical development of the topic, provided that they are familiar with other methodology, such as linear regression. Ultimately, one must engage the formulas for some of the linear curve estimators. These mathematical expressions for local smoothers are more easily understood after the student has seen a graph and a description of what the procedure is actually doing. In this article there are several such figures. These are mostly scatterplots of a single response against one predictor. Kernel smoothers have series expansions for bias and variance. The leading terms of those expansions yield approximate expressions for asymptotic mean squared error. In turn these provide one criterion for selection of the bandwidth. This choice of a smoothing parameter is done a rich variety of ways in practice. The final sections cover alternative approaches and extensions. The survey is supplemented with citations to some excellent books and articles. These provide the student with an entry into the literature, which is rapidly developing in traditional print media as well as on line.

##### MSC:
 62G08 Nonparametric regression and quantile regression
##### Software:
ElemStatLearn; fda (R); KernSmooth; SemiPar
Full Text:
##### References:
 [1] Altman, N. S. (1992). An introduction to kernel and nearest-neighbor nonparametric regression. Amer. Statist. 46 175–185. [2] Bartlett, M. S. (1963). Statistical estimation of density functions. Sankhyā Ser. A 25 245–254. · Zbl 0129.32302 [3] Benedetti, J. K. (1977). On the nonparametric estimation of regression functions. J. Roy. Statist. Soc. Ser. B 39 248–253. · Zbl 0367.62088 [4] Chu, C.-K. and Marron, J. S. (1991). Choosing a kernel regression estimator (with discussion). Statist. Sci. 6 404–436. JSTOR: · Zbl 0955.62561 [5] Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. J. Amer. Statist. Assoc. 74 829–836. · Zbl 0423.62029 [6] Epanechnikov, V. A. (1969). Nonparametric estimation of a multivariate probability density. Theory Probab. Appl. 14 153–158. · Zbl 0175.17101 [7] Eubank, R. L. (1999). Nonparametric Regression and Spline Smoothing , 2nd ed. Dekker, New York. · Zbl 0936.62044 [8] Fan, J. (1992). Design-adaptive nonparametric regression. J. Amer. Statist. Assoc. 87 998–1004. · Zbl 0850.62354 [9] Fan, J. and Gijbels, I. (1995). Data-driven bandwidth selection in local polynomial fitting: Variable bandwidth and spatial adaptation. J. Roy. Statist. Soc. Ser. B 57 371–394. · Zbl 0813.62033 [10] Fan, J. and Gijbels, I. (1996). Local Polynomial Modeling and Its Applications . Chapman and Hall, London. · Zbl 0873.62037 [11] Gasser, T. and Müller, H.-G. (1979). Kernel estimation of regression functions. Smoothing Techniques for Curve Estimation . Lecture Notes in Math. 757 23–68. Springer, Heidelberg. · Zbl 0418.62033 [12] Gerard, P. D. and Schucany, W. R. (1997). Locating exotherms in differential thermal analysis with nonparametric regression. J. Agric. Biol. Environ. Stat. 2 255–268. [13] Hart, J. D. (1997). Nonparametric Smoothing and Lack-of-Fit Tests . Springer, New York. · Zbl 0886.62043 [14] Hart, J. D. and Lee, C.-L. (2005). Robustness of one-sided cross-validation to autocorrelation. J. Multivariate Anal. 92 77–96. · Zbl 1065.62068 [15] Hastie, T., Tibshirani, R. and Friedman, J. (2001). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York. · Zbl 0973.62007 [16] Hodges, J. L., Jr. and Lehmann, E. L. (1956). The efficiency of some nonparametric competitors of the $$t$$-test. Ann. Math. Statist. 27 324–335. · Zbl 0075.29206 [17] Hurvich, C. M., Simonoff, J. S. and Tsai, C.-L. (1998). Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 271–293. · Zbl 0909.62039 [18] Hurvich, C. M. and Tsai, C.-L. (1989). Regression and time series model selection in small samples. Biometrika 76 297–307. · Zbl 0669.62085 [19] Jia, A. and Schucany, W. R. (2004). Recursive partitioning for kernel smoothers: A tree-based approach for estimating variable bandwidths in local linear regression. Unpublished manuscript. [20] Lin, X., Wang, N., Welsh, A. H. and Carroll, R. J. (2004). Equivalent kernels of smoothing splines in nonparametric regression for clustered/longitudinal data. Biometrika 91 177–193. · Zbl 1132.62321 [21] Loader, C. R. (1996). Change point estimation using nonparametric regression. Ann. Statist. 24 1667–1678. · Zbl 0867.62033 [22] Loader, C. R. (1999). Local Regression and Likelihood . Springer, New York. · Zbl 0929.62046 [23] Müller, H.-G. (1987). Weighted local regression and kernel methods for nonparametric curve fitting. J. Amer. Statist. Assoc. 82 231–238. · Zbl 0632.62039 [24] Müller, H.-G. (1992). Change-points in nonparametric regression analysis. Ann. Statist. 20 737–761. JSTOR: · Zbl 0783.62032 [25] Nadaraya, E. A. (1965). On nonparametric estimates of density functions and regression curves. Theory Probab. Appl. 10 186–190. · Zbl 0134.36302 [26] Parzen, E. (1962). On estimation of a probability density function and mode. Ann. Math. Statist. 33 1065–1076. · Zbl 0116.11302 [27] Pitblado, J. (2000). Estimating partially variable bandwidths in local linear regression using an information criterion. Ph.D. dissertation, Dept. Statistical Science, Southern Methodist Univ. [28] Priestley, M. B. and Chao, M. T. (1972). Nonparametric function fitting. J. Roy. Statist. Soc. Ser. B 34 385–392. · Zbl 0263.62044 [29] Ramsay, J. O. and Silverman, B. W. (1997). Functional Data Analysis . Springer, New York. · Zbl 0882.62002 [30] Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27 832–837. · Zbl 0073.14602 [31] Ruppert, D. (1997). Empirical-bias bandwidths for local polynomial nonparametric regression and density estimation. J. Amer. Statist. Assoc. 92 1049–1062. · Zbl 1067.62531 [32] Ruppert, D., Wand, M. P. and Carroll, R. J. (2003). Semiparametric Regression . Cambridge Univ. Press. · Zbl 1038.62042 [33] Schlee, W. (1988). Regressograms. Encyclopedia of Statistical Sciences 8 1–3. Wiley, New York. [34] Scott, D. W. (1992). Multivariate Density Estimation . Wiley, New York. · Zbl 0850.62006 [35] Sheather, S. (2005). Density estimation. Statist. Sci. 19 588–597. · Zbl 1100.62558 [36] Signorini, D. F. and Jones, M. C. (2004). Kernel estimators for univariate binary regression. J. Amer. Statist. Assoc. 99 119–126. · Zbl 1089.62506 [37] Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis . Chapman and Hall, London. · Zbl 0617.62042 [38] Spencer, J. (1904). On the graduation of rates of sickness and mortality. J. Institute of Actuaries 38 334–343. [39] Stone, C. J. (1977). Consistent nonparametric regression (with discussion). Ann. Statist. 5 595–645. JSTOR: · Zbl 0366.62051 [40] Tukey, J. W. (1961). Curves as parameters and touch estimation. Proc. Fourth Berkeley Symp. Math. Statist. Probab. 1 681–694. Univ. California Press, Berkeley. · Zbl 0105.12304 [41] Ullah, A. (1985). Specification analysis of econometric models. J. Quantitative Economics 1 187–209. [42] Wahba, G. and Wold, S. (1975). A completely automatic French curve. Comm. Statist. 4 1–17. · Zbl 0305.62043 [43] Wand, M. P. and Jones, M. C. (1995). Kernel Smoothing . Chapman and Hall, London. · Zbl 0854.62043 [44] Watson, G. S. (1964). Smooth regression analysis. Sankhyā Ser. A 26 359–372. · Zbl 0137.13002 [45] Welsh, A. H., Lin, X. and Carroll, R. J. (2002). Marginal longitudinal nonparametric regression: Locality and efficiency of spline and kernel methods. J. Amer. Statist. Assoc. 97 482–493. · Zbl 1073.62529
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.