Criteria for Bayesian model choice with application to variable selection. (English) Zbl 1257.62023

Summary: In objective Bayesian model selection, no single criterion has emerged as dominant in defining objective prior distributions. Indeed, many criteria have been separately proposed and utilized to propose differing prior choices. We first formalize the most general and compelling of the various criteria that have been suggested, together with a new criterion. We then illustrate the potential of these criteria in determining objective model selection priors by considering their application to the problem of variable selection in normal linear models. This results in a new model selection objective prior with a number of compelling properties.


62F15 Bayesian inference
62J05 Linear regression; mixed models
62H99 Multivariate analysis
62C10 Bayesian problems; characterization of Bayes procedures
Full Text: DOI arXiv Euclid


[1] Abramowitz, M. and Stegun, I. A. (1964). Handbook of Mathematical Functions with Formulas , Graphs , and Mathematical Tables . Dover, New York. · Zbl 0171.38503
[2] Bayarri, M. J. and García-Donato, G. (2008). Generalization of Jeffreys divergence-based priors for Bayesian hypothesis testing. J. R. Stat. Soc. Ser. B Stat. Methodol. 70 981-1003. · doi:10.1111/j.1467-9868.2008.00667.x
[3] Berger, J. (1980). A robust generalized Bayes estimator and confidence region for a multivariate normal mean. Ann. Statist. 8 716-761. · Zbl 0464.62026 · doi:10.1214/aos/1176345068
[4] Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis , 2nd ed. Springer, New York. · Zbl 0572.62008
[5] Berger, J. O., Bayarri, M. J. and Pericchi, L. R. (2012). The effective sample size. Econometric Reviews .
[6] Berger, J. O., Ghosh, J. K. and Mukhopadhyay, N. (2003). Approximations and consistency of Bayes factors as model dimension grows. J. Statist. Plann. Inference 112 241-258. · Zbl 1026.62018 · doi:10.1016/S0378-3758(02)00336-1
[7] Berger, J. O. and Pericchi, L. R. (1996). The intrinsic Bayes factor for model selection and prediction. J. Amer. Statist. Assoc. 91 109-122. · Zbl 0870.62021 · doi:10.2307/2291387
[8] Berger, J. O., Pericchi, L. R. and Varshavsky, J. A. (1998). Bayes factors and marginal distributions in invariant situations. Sankhyā Ser. A 60 307-321. · Zbl 0973.62017
[9] Berger, J. O. and Pericchi, L. R. (2001). Objective Bayesian methods for model selection: Introduction and comparison. In Model Selection. Institute of Mathematical Statistics Lecture Notes-Monograph Series 38 135-207. IMS, Beachwood, OH. · doi:10.1214/lnms/1215540968
[10] Casella, G., Girón, F. J., Martínez, M. L. and Moreno, E. (2009). Consistency of Bayesian procedures for variable selection. Ann. Statist. 37 1207-1228. · Zbl 1160.62004 · doi:10.1214/08-AOS606
[11] Cui, W. and George, E. I. (2008). Empirical Bayes vs. fully Bayes variable selection. J. Statist. Plann. Inference 138 888-900. · Zbl 1130.62007 · doi:10.1016/j.jspi.2007.02.011
[12] De Santis, F. and Spezzaferri, F. (1999). Methods for default and roubst Bayesian model comparison: The fractional Bayes factor approach. International Statistical Review 67 267-286. · Zbl 0944.62027 · doi:10.2307/1403706
[13] Fernández, C., Ley, E. and Steel, M. F. J. (2001). Benchmark priors for Bayesian model averaging. J. Econometrics 100 381-427. · Zbl 1091.62507 · doi:10.1016/S0304-4076(00)00076-2
[14] Forte, A. (2011). Objective Bayesian criteria for variable selection. Ph.D. thesis, Univ. de Valencia.
[15] Ghosh, J. K. and Samanta, T. (2002). Nonsubjective Bayes testing-an overview. J. Statist. Plann. Inference 103 205-223. · Zbl 0989.62017 · doi:10.1016/S0378-3758(01)00222-1
[16] Guo, R. and Speckman, P. L. (2009). Bayes factors consistency in linear models. Presented in O’Bayes 09 conference.
[17] Hsiao, C. K. (1997). Approximate Bayes factors when a mode occurs on the boundary. J. Amer. Statist. Assoc. 92 656-663. · Zbl 1067.62519 · doi:10.2307/2965713
[18] Jeffreys, H. (1961). Theory of Probability , 3rd ed. Clarendon, Oxford. · Zbl 0116.34904
[19] Kass, R. E. and Raftery, A. E. (1995). Bayes factors. J. Amer. Statist. Assoc. 90 773-795. · Zbl 0846.62028 · doi:10.2307/2291091
[20] Kass, R. E. and Vaidyanathan, S. K. (1992). Approximate Bayes factors and orthogonal parameters, with application to testing equality of two binomial proportions. J. Roy. Statist. Soc. Ser. B 54 129-144. · Zbl 0777.62032
[21] Kass, R. E. and Wasserman, L. (1995). A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J. Amer. Statist. Assoc. 90 928-934. · Zbl 0851.62020 · doi:10.2307/2291327
[22] Laud, P. W. and Ibrahim, J. G. (1995). Predictive model selection. J. Roy. Statist. Soc. Ser. B 57 247-262. · Zbl 0809.62024
[23] Liang, F., Paulo, R., Molina, G., Clyde, M. A. and Berger, J. O. (2008). Mixtures of \(g\) priors for Bayesian variable selection. J. Amer. Statist. Assoc. 103 410-423. · Zbl 1335.62026 · doi:10.1198/016214507000001337
[24] Maruyama, Y. and George, E. I. (2008). gBF: A fully Bayes factor with a generalized \(g\)-prior. Available at [stat.ME]. · Zbl 1231.62036 · doi:10.1214/11-AOS917
[25] Maruyama, Y. and Strawderman, W. E. (2010). Robust Bayesian variable selection with sub-harmonic priors. Available at [stat.ME].
[26] Moreno, E., Bertolino, F. and Racugno, W. (1998). An intrinsic limiting procedure for model selection and hypotheses testing. J. Amer. Statist. Assoc. 93 1451-1460. · Zbl 1064.62513 · doi:10.2307/2670059
[27] Pérez, J. M. and Berger, J. O. (2002). Expected-posterior prior distributions for model selection. Biometrika 89 491-511. · Zbl 1036.62026 · doi:10.1093/biomet/89.3.491
[28] Robert, C. P., Chopin, N. and Rousseau, J. (2009). Harold Jeffreys’s theory of probability revisited. Statist. Sci. 24 141-172. · Zbl 1328.62012 · doi:10.1214/09-STS284
[29] Scott, J. G. and Berger, J. O. (2010). Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. Ann. Statist. 38 2587-2619. · Zbl 1200.62020 · doi:10.1214/10-AOS792
[30] Spiegelhalter, D. J. and Smith, A. F. M. (1982). Bayes factors for linear and log-linear models with vague prior information. J. Roy. Statist. Soc. Ser. B 44 377-387. · Zbl 0502.62032
[31] Strawderman, W. E. (1971). Proper Bayes minimax estimators of the multivariate normal mean. Ann. Math. Statist. 42 385-388. · Zbl 0222.62006 · doi:10.1214/aoms/1177693528
[32] Suzuki, Y. (1983). On Bayesian approach to model selection. In Proceedings of the International Statistical Institute 288-291. ISI Publications, Voorburg.
[33] Weisstein, E. W. (2009). Appell hypergeometric function from mathworld-a Wolfram web resource. Available at .
[34] Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with \(g\)-prior distributions. In Bayesian Inference and Decision Techniques : Essays in Honor of Bruno de Finetti (A. Zellner, ed.) 389-399. North-Holland, Amsterdam. · Zbl 0655.62071
[35] Zellner, A. and Siow, A. (1980). Posterior odds ratio for selected regression hypotheses. In Bayesian Statistics 1 (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 585-603. Univeristy Press, Valencia. · Zbl 0457.62004
[36] Zellner, A. and Siow, A. (1984). Basic Issues in Econometrics . University of Chicago Press, Chicago.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.