×

zbMATH — the first resource for mathematics

Inference and computation with generalized additive models and their extensions. (English) Zbl 1447.62085
Summary: Regression models in which a response variable is related to smooth functions of some predictor variables are popular as a result of their appealing balance between flexibility and interpretability. Since the original generalized additive models of T. J. Hastie and R. J. Tibshirani [Generalized additive models. London etc.: Chapman and Hall (1990; Zbl 0747.62061)] numerous model extensions have been proposed, and a variety of practically useful computational strategies have emerged. This paper provides an overview of some widely applicable frameworks for this type of modelling, emphasizing the similarities between the different approaches, and the equivalence of smoothing, Gaussian latent process models and Gaussian random effects. The focus is particularly on Bayes empirical smoother theory, fully Bayesian inference via stochastic simulation or integrated nested Laplace approximation and boosting.

MSC:
62J05 Linear regression; mixed models
62J07 Ridge regression; shrinkage estimators (Lasso)
62J12 Generalized linear models (logistic models)
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Akaike, H.; Petran, B.; Csaaki, F., Information theory and an extension of the maximum likelihood principle, International symposium on information theory, 267-281 (1973), Budapest: Akadeemiai Kiadi, Budapest
[2] Anderssen, R.; Bloomfield, P., A time series approach to numerical differentiation, Technometrics, 16, 1, 69-75 (1974) · Zbl 0286.65012
[3] Augustin, NH; Sauleau, E-A; Wood, SN, On quantile quantile plots for generalized linear models, Comput Stat Data Anal, 56, 8, 2404-2409 (2012) · Zbl 1252.62072
[4] Belitz C, Brezger A, Kneib T, Lang S, Umlauf N (2015). BayesX: software for Bayesian inference in structured additive regression models
[5] Bissiri, PG; Holmes, C.; Walker, SG, A general framework for updating belief distributions, J R Stat Soc Ser B (Stat Methodol), 78, 5, 1103-1130 (2016) · Zbl 1414.62039
[6] Claeskens, G.; Krivobokova, T.; Opsomer, JD, Asymptotic properties of penalized spline estimators, Biometrika, 96, 3, 529-544 (2009) · Zbl 1170.62031
[7] Clarke, E.; Speirs, D.; Heath, M.; Wood, S.; Gurney, W.; Holmes, S., Calibrating remotely sensed chlorophyll—a data by using penalized regression splines, J R Stat Soc Ser C (Appl Stat), 55, 3, 331-353 (2006) · Zbl 05188740
[8] Craven, P.; Wahba, G., Smoothing noisy data with spline functions, Numer Math, 31, 377-403 (1979) · Zbl 0377.65007
[9] Davis, TA, Direct methods for sparse linear systems (2006), Philadelphia: SIAM, Philadelphia
[10] de Boor, C., A practical guide to splines (2001), New York: Springer, New York · Zbl 0987.65015
[11] Duchon, J.; Schemp, W.; Zeller, K., Splines minimizing rotation-invariant semi-norms in Solobev spaces, Construction theory of functions of several variables, 85-100 (1977), Berlin: Springer, Berlin
[12] Eilers, PH; Marx, BD; Durbán, M., Twenty years of p-splines, SORT Stat Oper Res Trans, 39, 2, 149-186 (2015) · Zbl 1339.41010
[13] Eilers, PHC; Marx, BD, Flexible smoothing with B-splines and penalties, Stat Sci, 11, 2, 89-121 (1996) · Zbl 0955.62562
[14] Fahrmeir, L.; Kneib, T.; Lang, S., Penalized structured additive regression for space-time data: a Bayesian perspective, Stat Sin, 14, 3, 731-761 (2004) · Zbl 1073.62025
[15] Fahrmeir, L.; Lang, S., Bayesian inference for generalized additive mixed models based on markov random field priors, Appl Stat, 50, 201-220 (2001)
[16] Fasiolo M, Goude Y, Nedellec R, Wood SN (2017) Fast calibrated additive quantile regression. arXiv preprint arXiv:1707.03307
[17] Fellner, WH, Robust estimation of variance components, Technometrics, 28, 1, 51-60 (1986) · Zbl 0597.62018
[18] Greven, S.; Kneib, T., On the behaviour of marginal and conditional AIC in linear mixed models, Biometrika, 97, 4, 773-789 (2010) · Zbl 1204.62114
[19] Gu, C., Cross-validating non-Gaussian data, J Comput Graph Stat, 1, 169-179 (1992)
[20] Gu, C., Smoothing spline ANOVA models (2013), New York: Springer, New York · Zbl 1269.62040
[21] Gu, C.; Wahba, G., Minimizing GCV/GML scores with multiple smoothing parameters via the Newton method, SIAM J Sci Stat Comput, 12, 2, 383-398 (1991) · Zbl 0727.65009
[22] Hastie, T.; Tibshirani, R., Generalized additive models (with discussion), Stati Sci, 1, 297-318 (1986) · Zbl 0645.62068
[23] Hastie, T.; Tibshirani, R., Generalized additive models (1990), Boca Raton: Chapman & Hall, Boca Raton · Zbl 0747.62061
[24] Kimeldorf, GS; Wahba, G., A correspondence between Bayesian estimation on stochastic processes and smoothing by splines, Ann Math Stat, 41, 2, 495-502 (1970) · Zbl 0193.45201
[25] Klein, N.; Kneib, T.; Klasen, S.; Lang, S., Bayesian structured additive distributional regression for multivariate responses, J R Stat Soc Seri C (Appl Stat), 64, 569-591 (2014)
[26] Klein, N.; Kneib, T.; Lang, S.; Sohn, A., Bayesian structured additive distributional regression with an application to regional income inequality in Germany, Ann Appl Stat, 9, 1024-1052 (2015) · Zbl 1454.62485
[27] Kristensen, K.; Nielsen, A.; Berg, CW; Bell, HSBM, TMB: automatic differentiation and laplaceapproximation, J Stat Softw, 70, 5, 1-21 (2016)
[28] Lang, S.; Brezger, A., Bayesian P-splines, J Comput Graph Stat, 13, 183-212 (2004)
[29] Lang, S.; Umlauf, N.; Wechselberger, P.; Harttgen, K.; Kneib, T., Multilevel structured additive regression, Stat Comput, 24, 2, 223-238 (2014) · Zbl 1325.62179
[30] Li, Z.; Wood, SN, Faster model matrix crossproducts for large generalized linear models with discretized covariates, Stat Comput, 30, 19-25 (2019) · Zbl 1436.62352
[31] Lindgren, F.; Rue, H.; Lindström, J., An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach, J R Stat Soc Ser B (Stat Methodol), 73, 4, 423-498 (2011) · Zbl 1274.62360
[32] Martins, TG; Simpson, D.; Lindgren, F.; Rue, H., Bayesian computing with INLA: new features, Comput Stat Data Anal, 67, 68-83 (2013) · Zbl 06970873
[33] Marx, BD; Eilers, PH, Multidimensional penalized signal regression, Technometrics, 47, 1, 13-22 (2005)
[34] Mayr, A.; Fenske, N.; Hofner, B.; Kneib, T.; Schmid, M., Generalized additive models for location, scale and shape for high dimensional data—a flexible approach based on boosting, J R Stat Soc Ser C (Appl Stat), 61, 3, 403-427 (2012)
[35] Nychka, D., Bayesian confidence intervals for smoothing splines, J Am Stat Assoc, 83, 404, 1134-1143 (1988)
[36] Parker, R.; Rice, J., Discussion of Silverman (1985), J R Stat Soc Ser B, 47, 1, 40-42 (1985)
[37] Plummer M (2003) JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling. In: Proceedings of the 3rd international workshop on distributed statistical computing (DSC 2003), pp 20-22
[38] Pya, N.; Wood, SN, Shape constrained additive models, Stat Comput, 25, 3, 543-559 (2015) · Zbl 1331.62367
[39] Rigby, R.; Stasinopoulos, DM, Generalized additive models for location, scale and shape, J R Stat Soc Ser C (Appl Stat), 54, 3, 507-554 (2005) · Zbl 05188697
[40] Robinzonov N, Hothorn T (2010) Boosting for estimating spatially structured additive models. In: Statistical modelling and regression structures. Springer, pp 181-196
[41] Rue, H.; Held, L., Gaussian Markov random fields: theory and applications (2005), Boca Raton: CRC Press, Boca Raton · Zbl 1093.60003
[42] Rue, H.; Martino, S.; Chopin, N., Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion), J R Stat Soc Ser B, 71, 2, 319-392 (2009) · Zbl 1248.62156
[43] Rue, H.; Riebler, A.; Sørbye, SH; Illian, JB; Simpson, DP; Lindgren, FK, Bayesian computing with INLA: a review, Ann Rev Stat Its Appl, 4, 395-421 (2017)
[44] Ruppert, D.; Wand, MP; Carroll, RJ, Semiparametric regression (2003), Cambridge: Cambridge University Press, Cambridge
[45] Schall, R., Estimation in generalized linear models with random effects, Biometrika, 78, 4, 719-727 (1991) · Zbl 0850.62561
[46] Schmid, M.; Hothorn, T., Boosting additive models using component-wise P-splines, Comput Stat Data Anal, 53, 2, 298-311 (2008) · Zbl 1231.62071
[47] Shun, Z.; McCullagh, P., Laplace approximation of high dimensional integrals, J R Stat Soc Ser B, 57, 4, 749-760 (1995) · Zbl 0826.41026
[48] Silverman, BW, Some aspects of the spline smoothing approach to non-parametric regression curve fitting, J R Stat Soc Ser B, 47, 1, 1-53 (1985)
[49] Spiegelhalter, DJ; Best, NG; Carlin, BP; van der Linde, A., Bayesian measures of model complexity and fit, J R Stat Soci Ser B, 64, 4, 583-639 (2002) · Zbl 1067.62010
[50] Stasinopoulos, DM; Rigby, RA, Generalized additive models for location scale and shape (gamlss) in r, J Stat Softw, 23, 7, 1-46 (2007)
[51] Stasinopoulos, MD; Rigby, RA; Heller, GZ; Voudouris, V.; De Bastiani, F., Flexible regression and smoothing: using GAMLSS in R (2017), Boca Raton: Chapman and Hall/CRC, Boca Raton
[52] Tutz, G.; Binder, H., Generalized additive modeling with implicit variable selection by likelihood-based boosting, Biometrics, 62, 4, 961-971 (2006) · Zbl 1116.62075
[53] Umlauf, N.; Adler, D.; Kneib, T.; Lang, S.; Zeileis, A., Structured additive regression models: an R interface to BayesX, J Stat Softw, 63, 21, 1-46 (2015)
[54] Wahba, G.; Cheney, E., Spline bases, regularization, and generalized cross validation for solving approximation problems with large quantities of noisy data, Approximation theory III (1980), London: Academic Press, London · Zbl 0485.41012
[55] Wahba, G., Bayesian confidence intervals for the cross validated smoothing spline, J R Stat Soc B, 45, 133-150 (1983) · Zbl 0538.65006
[56] Wahba, G., A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem, Ann Stat, 13, 4, 1378-1402 (1985) · Zbl 0596.65004
[57] Wahba, G., Spline models for observational data (1990), Philadelphia: SIAM, Philadelphia · Zbl 0813.62001
[58] Wood, SN, Modelling and smoothing parameter estimation with multiple quadratic penalties, J R Stat Soc Ser B (Stat Methodol), 62, 413-428 (2000)
[59] Wood, SN, Thin plate regression splines, J R Stat Soc Ser B (Stat Methodol), 65, 95-114 (2003) · Zbl 1063.62059
[60] Wood, SN, Fast stable direct fitting and smoothness selection for generalized additive models, J R Stat Soc Ser B (Stat Methodol), 70, 3, 495-518 (2008) · Zbl 05563356
[61] Wood, SN, Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models, J R Stat Soc Ser B (Stat Methodol), 73, 1, 3-36 (2011) · Zbl 1411.62089
[62] Wood, SN, On p-values for smooth components of an extended generalized additive model, Biometrika, 100, 1, 221-228 (2013) · Zbl 1284.62270
[63] Wood, SN, A simple test for random effects in regression models, Biometrika, 100, 4, 1005-1010 (2013) · Zbl 1279.62149
[64] Wood, SN, Core statistics (2015), Cambridge: Cambridge University Press, Cambridge
[65] Wood SN (2016) Just another Gibbs additive modeller: interfacing JAGS and mgcv. J Stati Softw 75(7)
[66] Wood, SN, Generalized additive models: an introduction with R (2017), Boca Raton: CRC Press, Boca Raton
[67] Wood, SN, P-splines with derivative based penalties and tensor product smoothing of unevenly distributed data, Stat Comput, 27, 4, 985-989 (2017) · Zbl 1384.65020
[68] Wood SN (2019) Simplified integrated nested laplace approximation. Biometrika (in press) · Zbl 1435.62341
[69] Wood, SN; Fasiolo, M., A generalized Fellner-Schall method for smoothing parameter optimization with application to Tweedie location, scale and shape models, Biometrics, 73, 4, 1071-1081 (2017) · Zbl 1405.62216
[70] Wood, SN; Li, Z.; Shaddick, G.; Augustin, NH, Generalized additive models for gigadata: modelling the UK black smoke network daily data, J Am Stat Assoc, 112, 519, 1199-1210 (2017)
[71] Wood, SN; Pya, N.; Säfken, B., Smoothing parameter and model selection for general smooth models (with discussion), J Am Stat Assoc, 111, 1548-1575 (2016)
[72] Yee, TW, Vector generalized linear and additive models: with an implementation in R (2015), Berlin: Springer, Berlin
[73] Yee, TW; Wild, C., Vector generalized additive models, J R Stat Soc Ser B (Methodol), 58, 3, 481-493 (1996) · Zbl 0855.62059
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.