×

zbMATH — the first resource for mathematics

Laplace approximations for fast Bayesian inference in generalized additive models based on P-splines. (English) Zbl 07345037
Summary: Generalized additive models (GAMs) are a well-established statistical tool for modeling complex nonlinear relationships between covariates and a response assumed to have a conditional distribution in the exponential family. To make inference in this model class, a fast and flexible approach is considered based on Bayesian P-splines and the Laplace approximation. The proposed Laplace-P-spline model contributes to the development of a new methodology to explore the posterior penalty space by considering a deterministic grid-based strategy or a Markov chain sampler, depending on the number of smooth additive terms in the predictor. The approach has the merit of relying on a simple Gaussian approximation to the conditional posterior of latent variables with closed form analytical expressions available for the gradient and Hessian of the approximate posterior penalty vector. This enables to construct accurate posterior pointwise and credible set estimators for (functions of) regression and spline parameters at a relatively low computational budget even for a large number of smooth additive components. The performance of the Laplace-P-spline model is confirmed through different simulation scenarios and the method is illustrated on two real datasets.
MSC:
62-XX Statistics
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Azzalini, A., A class of distributions which includes the normal ones, Scand. J. Stat., 12, 2, 171-178 (1985), www.jstor.org/stable/4615982 · Zbl 0581.62014
[2] Azzalini, A., The Skew-Normal and Related Families, Vol. 3 (2014), Cambridge University Press
[3] Bornkamp, B., Approximating probability densities by iterated Laplace approximations, J. Comput. Graph. Statist., 20, 3, 656-669 (2011)
[4] Comstock, G. W.; Bush, T. L.; Helzlsouer, K., Serum retinol, beta-carotene, vitamin E, and selenium as related to subsequent cancer of specific sites, Am. J. Epidemiol., 135, 2, 115-121 (1992)
[5] Eilers, P. H.C.; Marx, B. D., Flexible smoothing with B-splines and penalties, Stat. Sci., 11, 2, 89-121 (1996) · Zbl 0955.62562
[6] Eilers, P. H.C.; Marx, B. D.; Durbán, M., Twenty years of P-splines, SORT, 39, 2, 149-186 (2015) · Zbl 1339.41010
[7] Fraaije, R. G.A.; ter Braak, C. J.F.; Verduyn, B.; Breeman, L. B.S.; Verhoeven, J. T.A.; Soons, M. B., Early plant recruitment stages set the template for the development of vegetation patterns along a hydrological gradient, Funct. Ecol., 29, 7, 971-980 (2015)
[8] Gómez-Rubio, V.; Rue, H., Markov chain Monte Carlo with the Integrated Nested Laplace Approximation, Stat. Comput., 28, 5, 1033-1051 (2017) · Zbl 1405.62078
[9] Gressani, O.; Lambert, P., Fast Bayesian inference using Laplace approximations in a flexible promotion time cure model based on P-splines, Comput. Statist. Data Anal., 124, 151-167 (2018) · Zbl 06920885
[10] Gurmu, S., Semi-parametric estimation of hurdle regression models with an application to Medicaid utilization, J. Appl. Econom., 12, 3, 225-242 (1997)
[11] Hastie, T.; Tibshirani, R., Generalized additive models, Statist. Sci., 1, 3, 297-310 (1986) · Zbl 0645.62068
[12] Hastie, T.; Tibshirani, R., Generalized additive models: some applications, J. Amer. Statist. Assoc., 82, 398, 371-386 (1987) · Zbl 0633.62067
[13] Hastie, T. J.; Tibshirani, R. J., (Generalized Additive Models. Generalized Additive Models, Monographs on Statistics and Applied Probability, vol. 43 (1990), Chapman & Hall, London) · Zbl 0747.62061
[14] Hui, F. K.C.; You, C.; Shang, H. L.; Müller, S., Semiparametric regression using variational approximations, J. Amer. Statist. Assoc., 114, 528, 1765-1777 (2019) · Zbl 1428.62156
[15] Jullion, A.; Lambert, P., Robust specification of the roughness penalty prior distribution in spatially adaptive Bayesian P-splines models, Comput. Stat. Data Anal., 51, 5, 2542-2558 (2007) · Zbl 1161.62340
[16] Krivobokova, T.; Crainiceanu, C. M.; Kauermann, G., Fast adaptive penalized splines, J. Comput. Graph. Statist., 17, 1, 1-20 (2008), www.jstor.org/stable/27594289
[17] Lang, S.; Brezger, A., Bayesian P-splines, J. Comput. Graph. Stat., 13, 1, 183-212 (2004)
[18] Leonard, T., Comment on “A simple predictive density function,” by M. Lejeune and G.D. Faulkenberry, J. Amer. Statist. Assoc., 77, 379, 657-658 (1982)
[19] Liu, X.; Wang, L.; Liang, H., Estimation and variable selection for semiparametric additive partial linear models, Statist. Sinica, 21, 3, 1225-1248 (2011) · Zbl 1223.62020
[20] Luts, J.; Broderick, T.; Wand, M. P., Real-time semiparametric regression, J. Comput. Graph. Statist., 23, 3, 589-615 (2014)
[21] Marra, G.; Wood, S. N., Practical variable selection for generalized additive models, Comput. Statist. Data Anal., 55, 7, 2372-2387 (2011) · Zbl 1328.62475
[22] Martins, T. G.; Simpson, D.; Lindgren, F.; Rue, H., Bayesian computing with INLA: new features, Comput. Statist. Data Anal., 67, 68-83 (2013) · Zbl 06970873
[23] McCullagh, P.; Nelder, J. A., Generalized Linear Models, Vol. 37 (1989), CRC Press · Zbl 0744.62098
[24] Nelder, J. A.; Wedderburn, R. W., Generalized linear models, J. R. Stat. Soc. Ser. A, 135, 3, 370-384 (1972)
[25] Nierenberg, D. W.; Stukel, T. A.; Baron, J. A.; Dain, B. J.; Greenberg, E. R.; Group, S. C.P. S., Determinants of plasma levels of beta-carotene and retinol, Am. J. Epidemiol., 130, 3, 511-521 (1989)
[26] Rimm, E. B.; Stampfer, M. J.; Ascherio, A.; Giovannucci, E.; Colditz, G. A.; Willett, W. C., Vitamin E consumption and the risk of coronary heart disease in men, New Engl. J. Med., 328, 20, 1450-1456 (1993)
[27] Rue, H.; Martino, S.; Chopin, N., Approximate Bayesian inference for latent Gaussian models by using Integrated Nested Laplace Approximations, J. R. Stat. Soc. Ser. B, 71, 2, 319-392 (2009) · Zbl 1248.62156
[28] Rue, H.; Riebler, A.; Sørbye, S. H.; Illian, J. B.; Simpson, D. P.; Lindgren, F. K., Bayesian computing with INLA: a review, Annu. Rev. Stat. Appl., 4, 395-421 (2017)
[29] Ruppert, D.; Wand, M. P.; Carroll, R. J., Semiparametric Regression (2003), Cambridge University Press · Zbl 1038.62042
[30] Sapra, S. K., Generalized additive models in business and economics, Int. J. Adv. Stat. Probab., 1, 3 (2013)
[31] Stukel, T., Determinants of plasma retinol and beta-carotene levels (2008), StatLib Datasets Archive. URL http://lib.stat.cmu.edu/datasets/Plasma_Retinol
[32] Tierney, L.; Kadane, J. B., Accurate approximations for posterior moments and marginal densities, J. Amer. Statist. Assoc., 81, 393, 82-86 (1986) · Zbl 0587.62067
[33] Umlauf, N.; Adler, D.; Kneib, T.; Lang, S.; Zeileis, A., Structured additive regression models: An R interface to BayesX, J. Stat. Softw., 63, 21, 1-46 (2015)
[34] Ventrucci, M.; Rue, H., Penalized complexity priors for degrees of freedom in Bayesian P-splines, Stat. Model., 16, 6, 429-453 (2016)
[35] Wand, M. P., Fast approximate inference for arbitrarily large semiparametric regression models via message passing, J. Amer. Statist. Assoc., 112, 517, 137-168 (2017)
[36] Wand, M.; Ormerod, J., On semiparametric regression with O’Sullivan penalized splines, Aust. N.Z. J. Stat., 50, 2, 179-198 (2008) · Zbl 1146.62030
[37] Wood, S. N., Thin plate regression splines, J. R. Stat. Soc. Ser. B Stat. Methodol., 65, 1, 95-114 (2003) · Zbl 1063.62059
[38] Wood, S. N., Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models, J. R. Stat. Soc. Ser. B Stat. Methodol., 73, 1, 3-36 (2011) · Zbl 1411.62089
[39] Wood, S. N., Generalized Additive Models: An Introduction with R (2017), CRC Press · Zbl 1368.62004
[40] Wood, S. N.; Scheipl, F.; Faraway, J. J., Straightforward intermediate rank tensor product smoothing in mixed models, Stat. Comput., 23, 3, 341-360 (2013) · Zbl 1322.62197
[41] Yoon, J.W., Wilson, S.P., 2011. Inference for latent variable models with many hyperparameters. In: Proceedings of the 58th World Statistical Congress of the International Statistical Institute, Dublin.
[42] Zhang, S.; Hunter, D. J.; Forman, M. R.; Rosner, B. A.; Speizer, F. E.; Colditz, G. A.; Manson, J. E.; Hankinson, S. E.; Willett, W. C., Dietary carotenoids and vitamins A, C, and E and risk of breast cancer, J. Natl. Cancer Inst., 91, 6, 547-556 (1999)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.