×

Gaussian processes and limiting linear models. (English) Zbl 1452.62064

Summary: Gaussian processes retain the linear model either as a special case, or in the limit. We show how this relationship can be exploited when the data are at least partially linear. However from the perspective of the Bayesian posterior, the Gaussian processes which encode the linear model either have probability of nearly zero or are otherwise unattainable without the explicit construction of a prior with the limiting linear model in mind. We develop such a prior, and show that its practical benefits extend well beyond the computational and conceptual simplicity of the linear model. For example, linearity can be extracted on a per-dimension basis, or can be combined with treed partition models to yield a highly efficient nonstationary model. Our approach is demonstrated on synthetic and real datasets of varying linearity and dimensionality.

MSC:

62-08 Computational methods for problems pertaining to statistics
62M30 Inference from spatial processes

Software:

tgp; R
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Berger, J. O.; de Oliveira, V.; Sansó, B., Objective Bayesian analysis of spatially correlated data, Journal of the American Statistical Association, 96, 456, 1361-1374 (2001) · Zbl 1051.62095
[2] Bernstein, D., Matrix Mathematics (2005), Princeton University Press: Princeton University Press Princeton, NJ
[3] Breiman, L., Bagging predictors, Machine Learning, 24, 2, 123-140 (1996) · Zbl 0858.68080
[4] Calder, C. A., Dynamic factor process convolution models for multivariate spacetime data with application to air quality assessment, Environmental and Ecological Statistics, 14, 229-247 (2007)
[5] Chilés, J.; Delfiner, P., Geostatistics: Modeling Spatial Uncertainty (1999), John Wiley and Sons, Inc · Zbl 0922.62098
[6] Chipman, H.; George, E.; McCulloch, R., Bayesian CART model search (with discussion), Journal of the American Statistical Association, 93, 935-960 (1998)
[7] Chipman, H.; George, E.; McCulloch, R., Bayesian treed models, Machine Learning, 48, 303-324 (2002) · Zbl 0998.68072
[8] Chu, W.; Keerthi, S. S.; Ong, C. J., Bayesian support vector regression using a unified loss function, IEEE Transactions on Neural Networks, 15, 1, 29-44 (2004)
[9] Cressie, N., Statistics for Spatial Data (1991), John Wiley and Sons, Inc · Zbl 0799.62002
[10] Dey, D.; Müller, P.; Sinha, D., Practical Nonparametric and Semiparametric Bayesian Statistics (1998), Springer-Verlag New York, Inc: Springer-Verlag New York, Inc New York, NY, USA · Zbl 0893.00018
[11] Drucker, H.; Burges, C. J.C.; Kaufman, L.; Smola, A. J.; Vapnik, V., Support vector regression machines, (Advances in Neural Information Processing Systems (1996), MIT Press), 155-161
[12] Friedman, J. H., Multivariate adaptive regression splines, Annals of Statistics, 19, 1, 1-67 (1991) · Zbl 0765.62064
[13] George, E. I.; McCulloch, R. E., Variable selection via Gibbs sampling, Journal of the American Statistical Association, 88, 881-889 (1993)
[14] Geweke, J., Variable selection and model comparison in regression, (Bernardo, J.; Berger, J.; Dawid, A.; Smith, A., In Bayesian Statistics, vol. 5 (1996), Oxford Press), 609-620
[15] Gilks, W.; Richardson, S.; Spiegelhalter, D., Markov Chain Monte Carlo in Practice (1996), Chapman & Hall: Chapman & Hall London · Zbl 0832.00018
[16] Gilleland, E.; Nychka, D., Statistical models for monitoring and regulating ground-level ozone, Environmetrics, 16, 535-546 (2005)
[17] Golub, G. H.; Van Loan, C. F., Matrix Computations (1996), Johns Hopkins: Johns Hopkins Baltimore, MD · Zbl 0865.65009
[18] Gramacy, R. B., : An R package for bayesian nonstationary, semiparametric nonlinear regression and design by treed gaussian process models, Journal of Statistical Software, 19 (2007), 9
[19] Gramacy, R.B., Lee, H.K.H., 2008. Bayesian treed Gaussian process models with an application to computer modeling. Journal of the American Statistical Association (in press); Gramacy, R.B., Lee, H.K.H., 2008. Bayesian treed Gaussian process models with an application to computer modeling. Journal of the American Statistical Association (in press) · Zbl 1205.62218
[20] Gramacy, R.B., Taddy, M.A., 2008. tgp: Bayesian treed Gaussian process models. R package version 2.1-2; Gramacy, R.B., Taddy, M.A., 2008. tgp: Bayesian treed Gaussian process models. R package version 2.1-2
[21] Harrison, D.; Rubinfeld, D. L., Hedonic housing prices and the demand for clean air, Journal of Environmental Economics and Management, 5, 81-102 (1978) · Zbl 0375.90023
[22] Joseph, V. R.; Hung, Y.; Sudjianto, A., Blind kriging: A new method for developing metamodels, ASME Journal of Mechanical Design, 130, 031102-1-8 (2008)
[23] Kennedy, M.; O’Hagan, A., Bayesian calibration of computer models (with discussion), Journal of the Royal Statistical Society, Series B, 63, 425-464 (2001) · Zbl 1007.62021
[24] Neal, R., 1997. Monte Carlo implementation of Gaussian process models for Bayesian regression and classification. Tech. Rep. CRG-TR-97-2, Dept. of Computer Science, University of Toronto; Neal, R., 1997. Monte Carlo implementation of Gaussian process models for Bayesian regression and classification. Tech. Rep. CRG-TR-97-2, Dept. of Computer Science, University of Toronto
[25] Rasmussen, C.; Ghahramani, Z., Infinite mixtures of Gaussian process experts, (Advances in Neural Information Processing Systems, vol. 14 (2002), MIT Press), 881-888
[26] R Development Core Team, 2004. R : A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Aus. ISBN 3-900051-00-3; R Development Core Team, 2004. R : A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Aus. ISBN 3-900051-00-3
[27] Santner, T. J.; Williams, B. J.; Notz, W. I., The Design and Analysis of Computer Experiments (2003), Springer-Verlag: Springer-Verlag New York, NY · Zbl 1041.62068
[28] Silverman, B. W., Some aspects of the spline smoothing approach to non-parametric curve fitting, Journal of the Royal Statistical Society Series B, 47, 1-52 (1985) · Zbl 0606.62038
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.