×
Efron, Bradley

Selection criteria for scatterplot smoothers. (English) Zbl 1012.62040

Ann. Stat. 29, No. 2, 470-504 (2001).
Summary: Scatterplot smoothers estimate a regression function \(y=f(x)\) by local averaging of the observed data points \((x_i,y_i)\). In using a smoother, the statistician must choose a ‘window width’, a crucial smoothing parameter that says just how locally the averaging is done. This paper concerns the data-based choice of a smoothing parameter for splinelike smoothers, focusing on the comparison of two popular methods, \(C_p\) and generalized maximum likelihood. The latter is the MLE within a normal-theory empirical Bayes model.
We show that \(C_p\) is also maximum likelihood within a closely related non-normal family, both methods being examples of a class of selection criteria. Each member of the class is the MLE within its own one-parameter curved exponential family. Exponential family theory facilitates a finite-sample non-asymptotic comparison of the criteria. In particular it explains the eccentric behavior of \(C_p\), which even in favorable circumstances can easily select small window widths and wiggly estimates of \(f(x)\). The theory leads to simple geometric pictures of both \(C_p\) and MLE that are valid whether or not one believes in the probability models.

Cited in 1 Review
Cited in 20 Documents

MSC:

62G08 Nonparametric regression and quantile regression
62F10 Point estimation

Software:

gamsel
Cite Review PDF
Full Text: DOI

References:

[1] Anderssen, R. and Bloomfield, P. (1974). A time series approach to numerical differentiation. Technometrics 16 69-75. JSTOR: · Zbl 0286.65012 · doi:10.2307/1267494
[2] Efron, B. (1975). Defining the curvature of a statistical problem (with applications to second order efficiency). Ann. Statist. 3 1189-1242. · Zbl 0321.62013 · doi:10.1214/aos/1176343282
[3] Efron, B. (1978). The geometry of exponential families. Ann. Statist. 6 362-376. · Zbl 0436.62027 · doi:10.1214/aos/1176344130
[4] Efron, B. (1982). Maximum likelihood and decision theory. Ann. Statist. 10 340-356. · Zbl 0494.62004 · doi:10.1214/aos/1176345778
[5] Efron, B. (1986). How biased is the apparent rate of a prediction rule? J. Amer. Statist. Assoc. 8 461-470. JSTOR: · Zbl 0621.62073 · doi:10.2307/2289236
[6] Efron, B. (1999). Selection criteria for scatterplot smoothers. Technical Report 207, Stanford Univ. · Zbl 1012.62040
[7] Efron, B. (2000). Smoothers and the cost of model selection. Technical Report 209, Stanford Univ. · Zbl 1072.62570
[8] Efron, B. and Morris, C. (1973). Stein’s estimation rule and its competitors-an empirical Bayes approach. J. Amer. Statist. Assoc. 68 117-130. JSTOR: · Zbl 0275.62005 · doi:10.2307/2284155
[9] Efron, B. and Tibshirani, R. (1998). The problem of regions. Ann. Statist. 26 1687-1718. · Zbl 0954.62031 · doi:10.1214/aos/1024691353
[10] Eubank, R. (1988). Spline Smoothing and Nonparametric Regression. Dekker, New York. · Zbl 0702.62036
[11] Feller, W. (1971). An Introduction To Probability Theory and Its Applications 2. Wiley, New York. · Zbl 0219.60003
[12] Green, P. and Silverman, B. (1994). Nonparametric Regression and Generalized Linear Models. Chapman and Hall, London. · Zbl 0832.62032
[13] Hall, P. and Johnstone, I. (1992). Empirical functionals and efficient smoothing parameter selection. J. Roy. Statist. Soc. Ser. B 54 475-530. JSTOR: · Zbl 0786.62050
[14] Hastie, T. (1996). Pseudosplines. J. Roy. Statist. Soc. Ser. B 58 379-396. JSTOR: · Zbl 0853.62035
[15] Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London. · Zbl 0747.62061
[16] Li, K. (1986). Asymptotic optimality of CL and generalized cross-validation in ridge regression, with applications to spline smoothing. Ann. Statist. 14 1101-1112. · Zbl 0629.62043 · doi:10.1214/aos/1176350052
[17] Mallows, C. (1973). Some comments on Cp. Technometrics 15 661-667. · Zbl 0269.62061 · doi:10.2307/1267380
[18] Patterson, H. and Thompson, R. (1971). Recovery of inter-block information when block sizes are unequal. Biometrika 58 545-554. JSTOR: · Zbl 0228.62046 · doi:10.1093/biomet/58.3.545
[19] Stein, C. (1981). Estimating the mean of a multivariate normal distribution. Ann. Statist. 9 1135-1151. · Zbl 0476.62035 · doi:10.1214/aos/1176345632
[20] Stein, M. (1990). A comparison of generalized cross-validation and modified maximum likelihood for estimating the parameters of a stochastic process. Ann. Statist. 18 1139-1157. · Zbl 0734.62091 · doi:10.1214/aos/1176347743
[21] Wahba, G. (1985). A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem. Ann. Statist. 13 1378-1402. · Zbl 0596.65004 · doi:10.1214/aos/1176349743
[22] Wahba, G. (1990). Spline Models For Observational Data. SIAM, Philadelphia. · Zbl 0813.62001
[23] Wecker, W. and Ansley, C. (1983). The signal extraction approach to non-linear regression and spline smoothing. J. Amer. Statist. Assoc. 78 81-89. JSTOR: · Zbl 0536.62071 · doi:10.2307/2287113
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.