×

Misspecification in infinite-dimensional Bayesian statistics. (English) Zbl 1095.62031

Summary: We consider the asymptotic behavior of posterior distributions if the model is misspecified. Given a prior distribution and a random sample from a distribution \(P_0\), which may not be in the support of the prior, we show that the posterior concentrates its mass near the points in the support of the prior that minimize the Kullback-Leibler divergence with respect to \(P_0\). An entropy condition and a prior-mass condition determine the rate of convergence. The method is applied to several examples, with special interest for infinite-dimensional models. These include Gaussian mixtures, nonparametric regression and parametric models.

MSC:

62F15 Bayesian inference
62G08 Nonparametric regression and quantile regression
62G20 Asymptotic properties of nonparametric inference
62F05 Asymptotic properties of parametric tests
62B10 Statistical aspects of information-theoretic topics
62A01 Foundations and philosophical topics in statistics
62G07 Density estimation

References:

[1] Berk, R. H. (1966). Limiting behavior of posterior distributions when the model is incorrect. Ann. Math. Statist. 37 51–58. [Corrigendum 37 745–746.] · Zbl 0151.23802 · doi:10.1214/aoms/1177699597
[2] Birgé, L. (1983). Approximation dans les espaces métriques et théorie de l’estimation. Z. Wahrsch. Verw. Gebiete 65 181–238. · Zbl 0506.62026 · doi:10.1007/BF00532480
[3] Bunke, O. and Milhaud, X. (1998). Asymptotic behavior of Bayes estimates under possibly incorrect models. Ann. Statist. 26 617–644. · Zbl 0929.62022 · doi:10.1214/aos/1028144851
[4] Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1–67. · Zbl 0595.62022 · doi:10.1214/aos/1176349830
[5] Diaconis, P. and Freedman, D. (1986). On inconsistent Bayes estimates of location. Ann. Statist. 14 68–87. · Zbl 0595.62023 · doi:10.1214/aos/1176349843
[6] Ferguson, T. S. (1973). A Bayesian analysis of some non-parametric problems. Ann. Statist. 1 209–230. · Zbl 0255.62037 · doi:10.1214/aos/1176342360
[7] Ferguson, T. S. (1974). Prior distributions on spaces of probability measures. Ann. Statist. 2 615–629. · Zbl 0286.62008 · doi:10.1214/aos/1176342752
[8] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531. · Zbl 1105.62315 · doi:10.1214/aos/1016218228
[9] Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29 1233–1263. · Zbl 1043.62025 · doi:10.1214/aos/1013203452
[10] Le Cam, L. M. (1986). Asymptotic Methods in Statistical Decision Theory . Springer, New York. · Zbl 0605.62002
[11] Megginson, R. E. (1998). An Introduction to Banach Space Theory . Springer, New York. · Zbl 0910.46008 · doi:10.1007/978-1-4612-0603-3
[12] Pfanzagl, J. (1988). Consistency of maximum likelihood estimators for certain nonparametric families, in particular: Mixtures. J. Statist. Plann. Inference 19 137–158. · Zbl 0656.62044 · doi:10.1016/0378-3758(88)90069-9
[13] Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10–26. · Zbl 0158.17606 · doi:10.1007/BF00535479
[14] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714. · Zbl 1041.62022 · doi:10.1214/aos/1009210686
[15] Strasser, H. (1985). Mathematical Theory of Statistics . de Gruyter, Berlin. · Zbl 0594.62017 · doi:10.1515/9783110850826
[16] Wong, W. H. and Shen, X. (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLE’s. Ann. Statist. 23 339–362. · Zbl 0829.62002 · doi:10.1214/aos/1176324524
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.