## Consistency of Bayesian procedures for variable selection.(English)Zbl 1160.62004

Summary: It has long been known that for the comparison of pairwise nested models, a decision based on the Bayes factor produces a consistent model selector (in the frequentist sense). Here we go beyond the usual consistency for nested pairwise models, and show that for a wide class of prior distributions, including intrinsic priors, the corresponding Bayesian procedure for variable selection in normal regression is consistent in the entire class of normal linear models. We find that the asymptotics of the Bayes factors for intrinsic priors are equivalent to those of the Schwarz (BIC) criterion. Also, recall that the Jeffreys-Lindley paradox [D.V. Lindley, Biometrika 44, 187–192 (1957)] refers to the well-known fact that a point null hypothesis on the normal mean parameter is always accepted when the variance of the conjugate prior goes to infinity. This implies that some limiting forms of proper prior distributions are not necessarily suitable for testing problems. Intrinsic priors are limits of proper prior distributions, and for finite sample sizes they have been proved to behave extremely well for variable selection in regression; a consequence of our results is that for intrinsic priors Lindley’s paradox does not arise.

### MSC:

 62C10 Bayesian problems; characterization of Bayes procedures 62A01 Foundations and philosophical topics in statistics 62F05 Asymptotic properties of parametric tests 62F15 Bayesian inference

### Keywords:

Bayes factors; intrinsic priors; linear models; consistency
Full Text:

### References:

 [1] Abramowitz, M. and Stegun, I. A. (1972). Handbook of Mathematical Functions . U.S. Government Printing Office, Washington, DC. · Zbl 0543.33001 [2] Berger, J. O. and Bernardo, J. M. (1992). On the development of the reference prior method. In Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. David and A. F. M. Smith, eds.) 35-60. Oxford Univ. Press, London. [3] Berger, J. O. and Pericchi, L. R. (1996). The intrinsic Bayes factor for model selection and prediction. J. Amer. Statist. Assoc. 91 109-122. JSTOR: · Zbl 0870.62021 · doi:10.2307/2291387 [4] Berger, J. O. and Pericchi, L. R. (2004). Training samples in objective Bayesian model selection. Ann. Statist. 32 841-869. · Zbl 1092.62034 · doi:10.1214/009053604000000229 [5] Casella, G. and Moreno, E. (2006). Objective Bayesian variable selection. J. Amer. Statist. Assoc. 101 157-167. · Zbl 1118.62313 · doi:10.1198/016214505000000646 [6] Dawid, A. P. (1992). Prequential analysis, stochastic complexity and Bayesian inference. In Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. David and A. F. M. Smith, eds.) 109-125. Oxford Univ. Press, London. [7] Diaconis, P. and Friedman, D. (1986). On the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1-67. · Zbl 0595.62022 · doi:10.1214/aos/1176349830 [8] Girón, F. J., Martínez, M. L., Moreno, E. and Torres, F. (2006). Objective testing procedures in linear models. Calibration of the p-values. Scandinavian J. Statist. 33 765-784. · Zbl 1164.62322 · doi:10.1111/j.1467-9469.2006.00514.x [9] Girón, F. J., Moreno, E. and Casella, G. (2007). Objective Bayesian analysis of multiple changepoints for linear models (with discussion). In Bayesian Statistics 8 (J. M. Bernardo, M. J. Bayarri and J. O. Berger, eds.) 227-252. Oxford Univ. Press, London. · Zbl 1252.62019 [10] Girón, F. J., Moreno, E. and Martínez, M. L. (2006). An objective Bayesian procedure for variable selection in regression. In Advances on Distribution Theory , Order Statistics and Inference (N. Balakrishnan et al., eds.) 393-408. Birkhäuser, Boston. · Zbl 05196684 [11] Jeffreys, H. (1967). Theory of Probability . Oxford Univ. Press, London. · Zbl 0171.22502 [12] Kass, R. E. and Wasserman, L. (1995). A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J. Amer. Statist. Assoc. 90 928-934. JSTOR: · Zbl 0851.62020 · doi:10.2307/2291327 [13] Lindley, D. V. (1957). A statistical paradox. Biometrika 44 187-192. JSTOR: · Zbl 0084.35806 · doi:10.1093/biomet/44.1-2.179 [14] Little, R. J. (2006). Calibrated Bayes: A Bayes/frequentist roadmap. Amer. Statist. 60 213-223. · doi:10.1198/000313006X117837 [15] Moreno, E. and Girón, F. J. (2005). Consistency of Bayes factors for linear models. C. R. Acad. Sci. Paris Ser. I Math. 340 911-914. · Zbl 1065.62037 [16] Moreno, E. and Girón, F. J. (2008). Comparison of Bayesian objective procedures for variable selection in linear regression. Test 3 472-492. · Zbl 1367.62071 [17] Moreno, E., Bertolino, F. and Racugno, W. (1998). An intrinsic limiting procedure for model selection and hypothesis testing. J. Amer. Statist. Assoc. 93 1451-1460. JSTOR: · Zbl 1064.62513 · doi:10.2307/2670059 [18] O’Hagan, A. and Forster, J. (2004). Kendall’s Advanced Theory of Statistics : Vol. 2B : Bayesian Inference . Arnold, London. · Zbl 1058.62002 [19] Robert, C. P. (1993). A note on Jeffreys-Lindley paradox. Statist. Sinica 3 601-608. · Zbl 0823.62006 [20] Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461-464. · Zbl 0379.62005 · doi:10.1214/aos/1176344136 [21] Shao, J. (1997). An asymptotic theory for linear model selection. Statist. Sinica 7 221-264. · Zbl 1003.62527
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.