Bayesian adaptive smoothing splines using stochastic differential equations. (English) Zbl 1327.62234

Summary: The smoothing spline is one of the most popular curve-fitting methods, partly because of empirical evidence supporting its effectiveness and partly because of its elegant mathematical formulation. However, there are two obstacles that restrict the use of the smoothing spline in practical statistical work. Firstly, it becomes computationally prohibitive for large data sets because the number of basis functions roughly equals the sample size. Secondly, its global smoothing parameter can only provide a constant amount of smoothing, which often results in poor performances when estimating inhomogeneous functions. In this work, we introduce a class of adaptive smoothing spline models that is derived by solving certain stochastic differential equations with finite element methods. The solution extends the smoothing parameter to a continuous data-driven function, which is able to capture the change of the smoothness of the underlying process. The new model is Markovian, which makes Bayesian computation fast. A simulation study and real data example are presented to demonstrate the effectiveness of our method.


62G05 Nonparametric estimation
62F15 Bayesian inference
62G08 Nonparametric regression and quantile regression
60H10 Stochastic ordinary differential equations (aspects of stochastic analysis)
Full Text: DOI arXiv Euclid


[1] Abramovich, F. and Steinberg, D. M. (1996). Improved inference in nonparametric regression using \({L}_{k}\)-smoothing splines. Journal of Statistical Planning and Inference 49 , 327-341. · Zbl 0881.62043 · doi:10.1016/0378-3758(95)00021-6
[2] Baladandayuthapani, V., Mallick, B. K. and Carroll, R. J. (2005). Spatially adaptive Bayesian penalized regression splines (P-splines). Journal of Computational and Graphical Statistics 14 , 378-394. · doi:10.1198/106186005X47345
[3] Brezger, A. and Lang, S. (2006). Generalized structured additive regression based on Bayesian P-splines. Computational Statistics and Data Analysis 50 , 967-991. · Zbl 1431.62308
[4] Crainiceanu, C., Ruppert, D., Carroll, R., Adarsh, J. and Goodner, B. (2007). Spatially adaptive Penalized splines with heteroscedastic errors. Journal of Computational and Graphical Statistics 16 , 265-288. · doi:10.1198/106186007X208768
[5] Cummins, D. J., Filloon, T. G. and Nychka, D. (2001). Confidence intervals for nonparametric curve estimates: Toward more uniform pointwise coverage. Journal of the American Statistical Association 96 , 233-246. · Zbl 1015.62049 · doi:10.1198/016214501750332811
[6] Duchon, J. (1977). Splines minimizing rotation-invariant semi-norms in Sobolev spaces. In Constructive Theory of Functions of Several Variables (W. Schempp and K. Zeller, eds.), volume 571 of Lecture Notes in Mathematics , 85-100, Springer Berlin / Heidelberg, 10.1007/BFb0086566. · Zbl 0342.41012 · doi:10.1007/BFb0086566
[7] Eilers, P. and Marx, B. (1996). Flexible smoothing with B-splines and penalties (with discussion). Statistical Science 11 , 89-121. · Zbl 0955.62562 · doi:10.1214/ss/1038425655
[8] Eilers, P. H. C. and Marx, B. D. (2010). Splines, knots, and penalties. Wiley Interdisciplinary Reviews: Computational Statistics 2 , 637-653.
[9] Eubank, R. L. (1999). Nonparametric Regression and Spline Smoothing . Marcel Dekker Inc. · Zbl 0936.62044
[10] Fahrmeir, L. and Knorr-Held, L. (2000). Dynamic and semiparametric models. In Smoothing and regression: approaches, computation, and application (M. G. Schimek, ed.), 513-544, New York: Wiley. · Zbl 0980.62079
[11] Fahrmeir, L. and Lang, S. (2001). Bayesian inference for generalized additive mixed models based on Markov random field priors. Journal of the Royal Statistical Society, Series C: Applied Statistics 50 , 201-220. · doi:10.1111/1467-9876.00229
[12] Fahrmeir, L. and Tutz, G. (2001). Multivariate Statistical Modeling based on Generalized Linear Models . Berlin: Springer. · Zbl 0980.62052
[13] Fahrmeir, L. and Wagenpfeil, S. (1996). Smoothing hazard functions and time-varying effects in discrete duration and competing risks models. Journal of the American Statistical Association 91 , 1584-1594. · Zbl 0883.62098 · doi:10.2307/2291584
[14] Green, P. J. and Silverman, B. W. (1994). Nonparametric Regression and Generalized Linear Models: a Roughness Penalty Approach . Chapman & Hall Ltd. · Zbl 0832.62032
[15] Gu, C. (2002). Smoothing Spline ANOVA Models . Springer-Verlag Inc, New York. · Zbl 1051.62034
[16] Kimeldorf, G. S. and Wahba, G. (1970). A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Annals of Mathematical Statistics 41 , 495-502. · Zbl 0193.45201 · doi:10.1214/aoms/1177697089
[17] Krivobokova, T., Crainiceanu, C. M. and Kauermann, G. (2008). Fast Adaptive Penalized Splines. Journal of Computational and Graphical Statistics 17 , 1-20. · doi:10.1198/106186008X287328
[18] Lang, S. and Brezger, A. (2004). Bayesian P-splines. Journal of Computational and Graphical Statistics 13 , 183-212. · doi:10.1198/1061860043010
[19] Lang, S., Fronk, E. M. and Fahrmeir, L. (2002). Function estimation with locally adaptive dynamic models. Computational Statistics 17 , 479-499. · Zbl 1037.62035 · doi:10.1007/s001800200121
[20] Lindgren, F. and Rue, H. (2008). On the second-order random walk model for irregular locations. Scandinavian Journal of Statistics 35 , 691-700. · Zbl 1199.60276 · doi:10.1111/j.1467-9469.2008.00610.x
[21] Lindgren, F., Rue, H. and Lindström, J. (2011). An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach (with discussion). Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73 , 423-498. · Zbl 1274.62360 · doi:10.1111/j.1467-9868.2011.00777.x
[22] Luo, Z. and Wahba, G. (1997). Hybrid adaptive splines. Journal of the American Statistical Association 92 , 107-116. · Zbl 1090.62535 · doi:10.2307/2291454
[23] O’Sullivan, F. (1986). A statistical perspective on ill-posed inverse problems. Statistical Science 1 , 502-527. · Zbl 0625.62110 · doi:10.1214/ss/1177013525
[24] Pintore, A., Speckman, P. L. and Holmes, C. C. (2006). Spatially adaptive smoothing splines. Biometrika 93 , 113-125. · Zbl 1152.62331 · doi:10.1093/biomet/93.1.113
[25] Rue, H. and Held, L. (2005). Gaussian Markov Random Fields: Theory and Applications , volume 104 of Monographs on Statistics and Applied Probability . Chapman & Hall, London.
[26] Rue, H., Martino, S. and Chopin, N. (2009). Approximate Bayesian inference for latent Gaussian models using integrated nested Laplace approximations (with discussion). Journal of the Royal Statistical Society, Series B: Statistical Methodology 71 , 319-392. · Zbl 1248.62156 · doi:10.1111/j.1467-9868.2008.00700.x
[27] Ruppert, D. and Carroll, R. J. (2000). Spatially-adaptive penalties for spline fitting. Australian & New Zealand Journal of Statistics 42 , 205-223.
[28] Ruppert, D., Wand, M. and Carroll, R. (2003). Semiparametric Regression . Cambridge University Press, Cambridge. · Zbl 1038.62042 · doi:10.1017/CBO9780511755453
[29] Scheipl, F. and Kneib, T. (2009). Locally adaptive Bayesian P-splines with a normal-exponential-gamma prior. Computational Statistics and Data Analysis 53 , 3533-3552. · Zbl 1453.62191
[30] Simpson, D., Helton, K. and Lindgren, F. (2012). On the connection between O’Sullivan splines, continuous random walk models, and smoothing splines. Technical report, Norwegian University of Science and Technology.
[31] Speckman, P. L. and Sun, D. (2003). Fully Bayesian spline smoothing and intrinsic autoregressive priors. Biometrika 90 , 289-302. · Zbl 1034.62023 · doi:10.1093/biomet/90.2.289
[32] Staniswalis, J. G. (1989). Local bandwidth selection for kernel estimates. Journal of the American Statistical Association 84 , 284-288. · Zbl 0676.62039 · doi:10.2307/2289875
[33] Staniswalis, J. G. and Yandell, B. S. (1992). Locally adaptive smoothing splines. Journal of Statistical Computation and Simulation 43 , 45-53. · doi:10.1080/00949659208811427
[34] Wahba, G. (1978). Improper priors, spline smoothing and the problem of guarding against model errors in regression. Journal of the Royal Statistical Society, Series B: Statistical Methodology 40 , 364-372. · Zbl 0407.62048
[35] Wahba, G. (1990). Spline Models for Observational Data . SIAM [Society for Industrial and Applied Mathematics], Philadelphia. · Zbl 0813.62001
[36] Walsh, J. (1986). An introduction to stochastic partial differential equations. In École d’Été de Probabilités de Saint Flour XIV - 1984 (R. Carmona, H. Kesten and J. Walsh, eds.), volume 1180 of Lecture Notes in Mathematics , 265-439, Springer Berlin / Heidelberg, 10.1007/BFb0074920.
[37] Wand, M. P. and Ormerod, J. T. (2008). On semiparametric regression with O’Sullivan penalized splines. Australian and New Zealand Journal of Statistics 50 , 179-198. · Zbl 1146.62030 · doi:10.1111/j.1467-842X.2008.00507.x
[38] Wecker, W. and Ansley, C. (1983). The signal extraction approach to nonlinear regression and spline smoothing. Journal of the American Statistical Association 78 , 81-89. · Zbl 0536.62071 · doi:10.2307/2287113
[39] Wood, S. (2006). Generalized additive models: an introduction with R . CRC Press. · Zbl 1087.62082
[40] Yue, Y., Speckman, P. and Sun, D. (2012). Priors for Bayesian adaptive spline smoothing. Annals of the Institute of Statistical Mathematics 64 , 577-613, 10.1007/s10463-010-0321-6. · Zbl 1237.62037 · doi:10.1007/s10463-010-0321-6
[41] Yue, Y. and Speckman, P. L. (2010). Nonstationary spatial Gaussian Markov random fields. Journal of Computational and Graphical Statistics 19 , 96-116. · doi:10.1198/jcgs.2009.08124
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.