×
Spiegelhalter, David J.; Best, Nicola G.; Carlin, Bradley P.; van der Linde, Angelika

Bayesian measures of model complexity and fit (with discussion). (English) Zbl 1067.62010

J. R. Stat. Soc., Ser. B, Stat. Methodol. 64, No. 4, 583-639 (2002).
Summary: We consider the problem of comparing complex hierarchical models in which the number of parameters is not clearly defined. Using an information theoretic argument we derive a measure \(p_D\) for the effective number of parameters in a model as the difference between the posterior mean of the deviance and the deviance at the posterior means of the parameters of interest. In general, \(p_D\) approximately corresponds to the trace of the product of Fisher’s information and the posterior covariance, which in normal models is the trace of the ‘hat’ matrix projecting observations onto fitted values. Its properties in exponential families are explored. The posterior mean deviance is suggested as a Bayesian measure of fit or adequacy, and the contributions of individual observations to the fit and complexity can give rise to a diagnostic plot of deviance residuals against leverages. Adding \(p_D\) to the posterior mean deviance gives a deviance information criterion for comparing models, which is related to other information criteria and has an approximate decision theoretic justification. The procedure is illustrated in some examples, and comparisons are drawn with alternative Bayesian and classical proposals. Throughout it is emphasized that the quantities required are trivial to compute in a Markov chain Monte Carlo analysis.

Cited in 12 Reviews
Cited in 1182 Documents

MSC:

62C10 Bayesian problems; characterization of Bayes procedures
62B10 Statistical aspects of information-theoretic topics
62F15 Bayesian inference
62A01 Foundations and philosophical topics in statistics

Keywords:

Bayesian model comparison; deviance information criterion; effective number of parameters; hierarchical models; leverage; Markov chain Monte Carlo methods; model dimension
PDF BibTeX XML Cite
\textit{D. J. Spiegelhalter} et al., J. R. Stat. Soc., Ser. B, Stat. Methodol. 64, No. 4, 583--639 (2002; Zbl 1067.62010)
Full Text: DOI

References:

[1] Akaike H., Proc. 2nd Int. Symp. Information Theory pp 267– (1973)
[2] Andrews D. F., J. R. Statist. Soc. 36 pp 99– (1974)
[3] Berk R. H., Ann. Math. Statist. 37 pp 51– (1966)
[4] Bernardo J. M., Ann. Statist. 7 pp 686– (1979)
[5] Bernardo J. M., Bayesian Theory (1994)
[6] Besag J., J. R. Statist. Soc. 36 pp 192– (1974)
[7] DOI: 10.1191/147108201128168 · Zbl 1104.62023
[8] Box G. E. P., J. Am. Statist. Ass. 71 pp 791– (1976)
[9] Breslow N. E., J. Am. Statist. Ass. 88 pp 9– (1993)
[10] Brownlee K. A., Statistical Theory and Methodology in Science and Engineering (1965) · Zbl 0136.39203
[11] DOI: 10.1214/aos/1028144851 · Zbl 0929.62022
[12] Burnham K. P., Model Selection and Inference (1998) · Zbl 0920.62006
[13] Carlin B. P., Bayes and Empirical Bayes Methods for Data Analysis (2000) · Zbl 1017.62005
[14] Chib S., Biometrika 85 pp 347– (1998)
[15] Clayton D. G., Biometrics 43 pp 671– (1987)
[16] Dempster A. P., Proc. Conf. Foundational Questions in Statistical Inference pp 335– (1974)
[17] DOI: 10.1023/A:1018598421607
[18] DOI: 10.1023/A:1018554606586
[19] Efron B., J. Am. Statist. Ass. 81 pp 461– (1986)
[20] DOI: 10.1002/sim.702
[21] DOI: 10.1111/j.0006-341X.1999.01145.x · Zbl 1059.62644
[22] Eubank R. L., J. R. Statist. Soc. 47 pp 332– (1985)
[23] DOI: 10.1016/0167-7152(86)90101-X · Zbl 0605.62082
[24] Fitzmaurice G., Biometrika 80 pp 141– (1993)
[25] Gelfand A. E., J. R. Statist. Soc. 56 pp 501– (1994)
[26] Gelfand A. E., Appl. Statist. 49 pp 171– (2000)
[27] Gelfand A., Biometrika 85 pp 1– (1998)
[28] Gelfand A. E., Technical Report (2002)
[29] Gilks W. R., Markov Chain Monte Carlo in Practice (1996) · Zbl 0832.00018
[30] Gilks W. R., Biometrics 49 pp 441– (1993)
[31] Good I. J., Ann. Math. Statist. 27 pp 1130– (1956)
[32] Green P., J. Am. Statist. Ass. (2002)
[33] DOI: 10.1198/016214501753208780
[34] Hastie T., Generalized Additive Models (1990) · Zbl 0747.62061
[35] Hodges J., Biometrika 88 pp 367– (2001)
[36] Huber P. J., Proc. 5th Berkeley Symp. Mathematical Statistics and Probability pp 221– (1967)
[37] Kass R., J. Am. Statist. Ass. 90 pp 773– (1995)
[38] Key J. T., Bayesian Statistics 6 pp 343– (1999)
[39] Kimeldorf G., Ann. Math. Statist. 41 pp 495– (1970)
[40] Kullback S., Ann. Math. Statist. 22 pp 79– (1951)
[41] Laird N. M., Biometrics 38 pp 963– (1982)
[42] Laud P. W., J. R. Statist. Soc. 57 pp 247– (1995)
[43] Lee Y., J. R. Statist. Soc. 58 pp 619– (1996)
[44] Linde A., Test 4 pp 63– (1995)
[45] DOI: 10.1016/S0378-3758(00)00116-6 · Zbl 0964.62060
[46] Lindley D. V., J. R. Statist. Soc. 34 pp 1– (1972)
[47] MacKay D. J. C., Neur. Computn 4 pp 415– (1992)
[48] DOI: 10.1088/0954-898X/6/3/011 · Zbl 0834.68098
[49] McCullagh P., Generalized Linear Models (1989) · Zbl 0744.62098
[50] Meng X.-L., Biometrika 79 pp 103– (1992)
[51] Moody J. E., Advances in Neural Information Processing Systems 4 pp 847– (1992)
[52] DOI: 10.1109/72.329683
[53] Natarajan R., J. Am. Statist. Ass. 95 pp 227– (2000)
[54] Raghunathan T. E., Technical Report (1988)
[55] Rahman N. J., Statist. Meth. Med. Res. 8 pp 195– (1999)
[56] DOI: 10.1111/1467-9868.00095
[57] Ripley B. D., Pattern Recognition and Neural Networks (1996) · Zbl 0853.62046
[58] Sawa T., Econometrica 46 pp 1273– (1978)
[59] Schwarz G., Ann. Statist. 6 pp 461– (1978)
[60] Slate E., J. Am. Statist. Ass. 89 pp 1471– (1994)
[61] Spiegelhalter D. J., WinBUGS Version 1.3 User Manual (2000)
[62] Spiegelhalter D. J., BUGS Examples Volume 1, Version 0.5 (Version ii) (1996)
[63] Stone M., J. R. Statist. Soc. 39 pp 44– (1977)
[64] Takeuchi K., Suri-Kagaku 153 pp 12– (1976)
[65] Vehtari A., IJCNN’99: Proc. 1999. Int. Joint Conf. Neural Networks (1999)
[66] Wahba G., J. R. Statist. Soc. 40 pp 364– (1978)
[67] J. R. Statist. Soc. 45 pp 133– (1983)
[68] Spline Models for Observational Data (1990) · Zbl 0813.62001
[69] Ye J., J. Am. Statist. Ass. 93 pp 120– (1998)
[70] Ye J., Technical Report (1998)
[71] Zeger S. L., J. Am. Statist. Ass. 86 pp 79– (1991)
[72] DOI: 10.1002/1097-0258(20000915/30)19:17/18<2265::AID-SIM568>3.0.CO;2-6
[73] Aitkin M., J. R. Statist. Soc. 53 pp 111– (1991)
[74] Akaike H., Proc. 2nd Int. Symp. Information Theory pp 267– (1973)
[75] Atkinson A. C., Biometrika 67 pp 413– (1980)
[76] Atkinson A. C., Robust Diagnostic Regression Analysis (2000) · Zbl 0964.62063
[77] Technical Report LSERR73 (2002)
[78] Bernardo J. M., Ann. Statist. 7 pp 686– (1979)
[79] Bernardo J. M., Bayesian Statistics 6 pp 101– (1999)
[80] Bernardo J. M., Bayesian Theory (1994)
[81] Bernardo J. M., 7th Valencia Int. Meet. Bayesian Statistics, Tenerife, June (2002)
[82] Burnham K. P., Model Selection and Inference: a Practical Information-theoretic Approach (1998) · Zbl 0920.62006
[83] Model Selection and Multimodel Inference: a Practical Information-theoretical Approach (2002) · Zbl 1005.62007
[84] G. Casella, C. P. Robert, and M. T. Wells (2000 ) Mixture models, latent variables and partitioned importance sampling .TechnicalReport. Paris. · Zbl 1075.65016
[85] Celeux G., J. Am. Statist. Ass. 95 pp 957– (2000)
[86] Cooke R. M., Experts in Uncertainty (1991)
[87] Cowell R. G., Probabilistic Networks and Expert Systems (1999) · Zbl 0937.68121
[88] Daniels M. J., J. Am. Statist. Ass. 94 pp 1254– (1999)
[89] Biometrics 57 pp 1173– (2001) · Zbl 0977.68617
[90] Dawid A. P., J. R. Statist. Soc. 147 pp 278– (1984)
[91] Kotz S., Encyclopedia of Statistical Sciences pp 210– (1986)
[92] J. R. Statist. Soc. 53 pp 79– (1991)
[93] Bernardo J. M., Bayesian Statistics 4 pp 109– (1992)
[94] Ghosh M., Current Issues in Statistical Inference: Essays in Honor of D. Basu pp 113– (1992) · Zbl 0760.00006
[95] Draper D., Bayesian Statistics 6 pp 541– (1999)
[96] Draper D., J. Global Optimzn 18 pp 399– (2000)
[97] Dupuis J., J. Statist. Planng Inf. (2002)
[98] Efron B., J. Am. Statist. Ass. 81 pp 461– (1986)
[99] D. Fouskakis, and D. Draper (2002 ) Stochastic optimization: a review . ,to be published. · Zbl 1217.62003
[100] Gangnon R., Spatial Cluster Modelling (2002)
[101] Gelfand A. E., Markov Chain Monte Carlo in Practice pp 145– (1996)
[102] Gelfand A. E., Bayesian Statistics 4 pp 147– (1992)
[103] Gelman A., Statist. Sin. 6 pp 733– (1996)
[104] Good I. J., J. R. Statist. Soc. 14 pp 107– (1952)
[105] Green P., J. Am. Statist. Ass. (2002)
[106] Hodges J., Biometrika 88 pp 367– (2001)
[107] Holmes C., Bayesian Statistics 6 pp 769– (1999)
[108] Kass R., J. Am. Statist. Ass. 90 pp 773– (1995)
[109] Key J. T., Bayesian Statistics 6 pp 343– (1999)
[110] R. King (2001 ) Bayesian model discrimination in the analysis of capture-recapture and related data .PhD Thesis. School of Mathematics, University of Bristol, Bristol.
[111] King R., Biometrika 88 pp 317– (2001)
[112] Konishi S., Biometrika 83 pp 875– (1996)
[113] Lauritzen S. L., J. R. Statist. Soc. 50 pp 157– (1988)
[114] DOI: 10.1002/1097-0258(20000915/30)19:17/18<2361::AID-SIM575>3.3.CO;2-E
[115] Lee Y., J. R. Statist. Soc. 58 pp 619– (1996)
[116] Biometrika 88 pp 987– (2001) · Zbl 1059.00007
[117] Statist. Modlng 1 pp 3– (2001) · Zbl 1004.62080
[118] Luna X., Scand. J. Statist. (2003)
[119] D. Madigan, and A. E. Raftery (1991 ) Model selection and accounting for model uncertainty in graphical models using Occam’s window .Technical Report 213. Department of Statistics, University of Washington, Seattle. · Zbl 0814.62030
[120] McKeague I., Spatial Cluster Modelling (2002)
[121] Meng X.-L., Biometrika 79 pp 103– (1992)
[122] Moreno E., Decision Research from Bayesian Approaches to Normative Systems (1998)
[123] Neter J., Applied Linear Statistical Models (1996)
[124] Pericchi L. R., Int. Statist. Rev. 58 pp 1– (1991)
[125] M. Plummer (2002 ) Some criteria for Bayesian model choice .Preprint. (Available from http://calvin.iarc.fr/martyn/papers/.)
[126] Priestley M. B., Spectral Analysis and Time Series (1981) · Zbl 0537.62075
[127] Robert C. P., Theory Decsn 40 pp 191– (1996)
[128] Shao J., Statist. Sin. 7 pp 221– (1997)
[129] Skouras K., Biometrika 86 pp 765– (1999)
[130] - (2000 ) Consistency in misspecified models . Research Report 218. Department of Statistical Science, University College London, London. (Available from: http://www.ucl.ac.uk/Stats/research/abs00.html#218.)
[131] Smith J. Q., Bayesian Statistics 5 pp 387– (1996)
[132] Stone M., J. R. Statist. Soc. 36 pp 111– (1974)
[133] J. R. Statist. Soc. 36 pp 44– (1977)
[134] A. Vehtari (2001 ) Bayesian model assessment and selection using expected utilities . DSc Dissertation. Helsinki University of Technology, Helsinki. (Available from http://lib.hut.fi/Diss/2001/isbn9512257653/.)
[135] Vehtari A., Neur. Computn 14 (2002)
[136] - (2002 ) Cross-validation, information criteria, expected utilities and the effective number of parameters . To be published.
[137] Volinsky C. T., Biometrics 56 pp 256– (2000)
[138] Weisberg S., Technometrics 23 pp 27– (1981)
[139] Ye J., J. Am. Statist. Ass. 93 pp 120– (1998)
[140] DOI: 10.1002/1097-0258(20000915/30)19:17/18<2265::AID-SIM568>3.0.CO;2-6
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.