From finite sample to asymptotics: a geometric bridge for selection criteria in spline regression. (English) Zbl 1076.62039

Summary: This paper studies, under the setting of spline regression, the connection between finite-sample properties of selection criteria and their asymptotic counterparts, focusing on bridging the gap between the two. We introduce a bias-variance decomposition of the prediction error, using which it is shown that in the asymptotics the bias term dominates the variability term, providing an explanation of the gap. A geometric exposition is provided for intuitive understanding. The theoretical and geometric results are illustrated through a numerical example.


62G08 Nonparametric regression and quantile regression
62G20 Asymptotic properties of nonparametric inference
62H12 Estimation in multivariate analysis
65C60 Computational problems in statistics (MSC2010)


Full Text: DOI arXiv


[1] Abramowitz, M. and Stegun, I. (1972). Handbook of Mathematical Functions , 10th printing. National Bureau of Standards, Washington. · Zbl 0543.33001
[2] Bowman, A. and Azzalini, A. (1997). Applied Smoothing Techniques for Data Analysis : The Kernel Approach with S-Plus Illustrations. Oxford Univ. Press, New York. · Zbl 0889.62027
[3] Demmler, A. and Reinsch, C. (1975). Oscillation matrices with spline smoothing. Numer. Math. 24 375–382. · Zbl 0297.65002
[4] Efron, B. (1975). Defining the curvature of a statistical problem (with application to second order efficiency) (with discussion). Ann. Statist. 3 1189–1242. · Zbl 0321.62013
[5] Efron, B. (2001). Selection criteria for scatterplot smoothers. Ann. Statist. 29 470–504. · Zbl 1012.62040
[6] Eubank, R. (1988). Spline Smoothing and Nonparametric Regression . Dekker, New York. · Zbl 0702.62036
[7] Fan, J. (2000). Prospects of nonparametric modeling. J. Amer. Statist. Assoc. 95 1296–1300. · Zbl 1072.62582
[8] Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications . Chapman and Hall, London. · Zbl 0873.62037
[9] Feller, W. (1971). An Introduction to Probability Theory and Its Applications 2 , 2nd ed. Wiley, New York. · Zbl 0219.60003
[10] Green, P. and Silverman, B. (1994). Nonparametric Regression and Generalized Linear Models . Chapman and Hall, London. · Zbl 0832.62032
[11] Gu, C. (1998). Model indexing and smoothing parameter selection in nonparametric function estimation (with discussion). Statist. Sinica 8 607–646. · Zbl 0901.62058
[12] Hall, P. and Johnstone, I. (1992). Empirical functionals and efficient smoothing parameter selection (with discussion). J. Roy. Statist. Soc. Ser. B 54 475–530. · Zbl 0786.62050
[13] Härdle, W. (1990). Applied Nonparametric Regression . Cambridge Univ. Press. · Zbl 0714.62030
[14] Härdle, W., Hall, P. and Marron, J. (1988). How far are automatically chosen regression smoothing parameters from their optimum (with discussion)? J. Amer. Statist. Assoc. 83 86–101. · Zbl 0644.62048
[15] Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models . Chapman and Hall, London. · Zbl 0747.62061
[16] Hurvich, C., Simonoff, J. and Tsai, C. (1998). Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 271–294. · Zbl 0909.62039
[17] Jones, M., Marron, J. and Sheather, S. (1996). A brief survey of bandwidth selection for density estimation. J. Amer. Statist. Assoc. 91 401–407. · Zbl 0873.62040
[18] Kohn, R., Ansley, C. and Tharm, D. (1991). The performance of cross-validation and maximum likelihood estimators of spline smoothing parameters. J. Amer. Statist. Assoc. 86 1042–1050.
[19] Kou, S. C. (2003). On the efficiency of selection criteria in spline regression. Probab. Theory Related Fields 127 153–176. · Zbl 1027.62021
[20] Kou, S. C. and Efron, B. (2002). Smoothers and the \(C_p\), generalized maximum likelihood and extended exponential criteria: A geometric approach. J. Amer. Statist. Assoc. 97 766–782. · Zbl 1048.62044
[21] Li, K.-C. (1986). Asymptotic optimality of \(C_L\) and generalized cross-validation in ridge regression with application to spline smoothing. Ann. Statist. 14 1101–1112. JSTOR: · Zbl 0629.62043
[22] Li, K.-C. (1987). Asymptotic optimality for \(C_p\), \(C_L\), cross-validation and generalized cross-validation: Discrete index set. Ann. Statist. 15 958–975. JSTOR: · Zbl 0653.62037
[23] Rosenblatt, M. (1991). Stochastic Curve Estimation . IMS, Hayward, CA. · Zbl 1163.62318
[24] Silverman, B. (1985). Some aspects of the spline smoothing approach to nonparametric regression curve fitting (with discussion). J. Roy. Statist. Soc. Ser. B 47 1–52. · Zbl 0606.62038
[25] Simonoff, J. (1996). Smoothing Methods in Statistics . Springer, New York. · Zbl 0859.62035
[26] Speckman, P. (1983). Efficient nonparametric regression with cross-validated smoothing splines. Unpublished manuscript.
[27] Speckman, P. (1985). Spline smoothing and optimal rates of convergence in nonparametric regression models. Ann. Statist. 13 970–983. JSTOR: · Zbl 0585.62074
[28] Speckman, P. and Sun, D. (2001). Asymptotic properties of smoothing parameter selection in spline regression.
[29] Stein, M. (1990). A comparison of generalized cross validation and modified maximum likelihood for estimating the parameters of a stochastic process. Ann. Statist. 18 1139–1157. JSTOR: · Zbl 0734.62091
[30] Wahba, G. (1985). A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem. Ann. Statist. 13 1378–1402. JSTOR: · Zbl 0596.65004
[31] Wahba, G. (1990). Spline Models for Observational Data . SIAM, Philadelphia. · Zbl 0813.62001
[32] Wecker, W. and Ansley, C. (1983). The signal extraction approach to nonlinear regression and spline smoothing. J. Amer. Statist. Assoc. 78 81–89. · Zbl 0536.62071
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.