×

Smoothing spline Gaussian regression: more scalable computation via efficient approximation. (English) Zbl 1062.62067

Summary: Smoothing splines via the penalized least squares method provide versatile and effective nonparametric models for regression with Gaussian responses. The computation of smoothing splines is generally of the order \(O(n^3)\), \(n\) being the sample size, which severely limits its practical applicability. We study more scalable computation of smoothing spline regression via certain low dimensional approximations that are asymptotically as efficient. A simple algorithm is presented and the Bayes model that is associated with the approximations is derived, with the latter guiding the porting of Bayesian confidence intervals. The practical choice of the dimension of the approximating space is determined through simulation studies, and empirical comparisons of the approximations with the exact solution are presented. Also evaluated is a simple modification of the generalized cross-validation method for smoothing parameter selection, which to a large extent fixes the occasional undersmoothing problem that is suffered by generalized cross-validation.

MSC:

62G08 Nonparametric regression and quantile regression
65C60 Computational problems in statistics (MSC2010)
62G20 Asymptotic properties of nonparametric inference
62F15 Bayesian inference

Software:

R; assist; FUNFITS; gss
PDF BibTeX XML Cite
Full Text: DOI Link

References:

[1] Breiman L., J. Am. Statist. Ass. 80 pp 580– (1985)
[2] Buja A., Ann. Statist. 17 pp 453– (1989)
[3] Craven P., Numer. Math. 31 pp 377– (1979)
[4] Dennis J. E., Numerical Methods for Unconstrained Optimization and Nonlinear Equations (1996) · Zbl 0847.65038
[5] Golub G., Matrix Computations (1989) · Zbl 0733.65016
[6] Green P. J., Nonparametric Regression and Generalized Linear Models (1994) · Zbl 0832.62032
[7] Gu C., Smoothing Spline ANOVA Models (2002) · Zbl 1051.62034
[8] Gu C., SIAM J. Matrix Anal. Applic. 10 pp 457– (1989)
[9] Gu C., Can. J. Statist. 30 pp 619– (2002)
[10] Gu C., SIAM J. Sci. Statist. Comput. 12 pp 383– (1991)
[11] Gu C., J. Comput. Graph. Statist. 2 pp 97– (1993)
[12] Gu C., J. R. Statist. Soc. 55 pp 353– (1993)
[13] Gu C., Statist. Sin. 13 pp 811– (2003)
[14] Ihaka R., J. Comput. Graph. Statist. 5 pp 299– (1996)
[15] Kimeldorf G., Ann. Math. Statist. 41 pp 495– (1970)
[16] Kimeldorf G., Sankhya 32 pp 173– (1970)
[17] Kimeldorf G., J. Math. Anal. Applic. 33 pp 82– (1971)
[18] Li K.-C., Ann. Statist. 14 pp 1101– (1986)
[19] Luo Z., J. Am. Statist. Ass. 92 pp 107– (1997)
[20] Nychka D., J. Am. Statist. Ass. 83 pp 1134– (1988)
[21] Nychka D., Case Studies in Environmental Statistics pp 159– (1998)
[22] Rao C. R., Linear Statistical Inference and Its Applications (1973) · Zbl 0256.62002
[23] Wahba G., J. R. Statist. Soc. 40 pp 364– (1978)
[24] Wahba G., SIAM J. Sci. Statist. Comput. 2 pp 5– (1981)
[25] Wahba G., J. R. Statist. Soc. 45 pp 133– (1983)
[26] Wahba G., Ann. Statist. 13 pp 1378– (1985)
[27] Wahba G., Spline Models for Observational Data (1990) · Zbl 0813.62001
[28] Wang Y., ASSIST: a Suite of S-plus Functions Implementing Spline Smoothing Techniques (2002)
[29] DOI: 10.1111/1467-9868.00240 · Zbl 04558581
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.