×

On the geometry of Bayesian inference. (English) Zbl 1435.62106

Summary: We provide a geometric interpretation to Bayesian inference that allows us to introduce a natural measure of the level of agreement between priors, likelihoods, and posteriors. The starting point for the construction of our geometry is the observation that the marginal likelihood can be regarded as an inner product between the prior and the likelihood. A key concept in our geometry is that of compatibility, a measure which is based on the same construction principles as Pearson correlation, but which can be used to assess how much the prior agrees with the likelihood, to gauge the sensitivity of the posterior to the prior, and to quantify the coherency of the opinions of two experts. Estimators for all the quantities involved in our geometric setup are discussed, which can be directly computed from the posterior simulation output. Some examples are used to illustrate our methods, including data related to on-the-job drug usage, midge wing length, and prostate cancer.

MSC:

62F15 Bayesian inference
60D05 Geometric probability and stochastic geometry
62P10 Applications of statistics to biology and medical sciences; meta analysis
62A01 Foundations and philosophical topics in statistics
PDF BibTeX XML Cite
Full Text: DOI arXiv Euclid

References:

[1] Agarawal, A. and Daumé, III, H. (2010). “A geometric view of conjugate priors.” Machine Learning 81, 99-113.
[2] Aitchison, J. (1971). “A geometrical version of Bayes’ theorem.” The American Statistician 25, 45-46.
[3] Al Labadi, L. and Evans, M. (2016). “Optimal robustness results for relative belief inferences and the relationship to prior-data conflict.” Bayesian Analysis 12, 705-728. · Zbl 1384.62076
[4] Amari, S.-i. (2016). Information Geometry and its Applications. New York: Springer. · Zbl 1350.94001
[5] Anaya-Izquierdo, K. and Marriott, P. (2007). “Local mixtures of the exponential distribution.” Annals of the Institute of Statistical Mathematics 59 111-134. · Zbl 1108.62018
[6] Berger, J. (1991). “Robust Bayesian analysis: Sensitivity to the prior.” Journal of Statistical Planning and Inference 25, 303-328. · Zbl 0747.62029
[7] Berger, J. and Berliner, L. M. (1986). “Robust Bayes and empirical Bayes analysis with \(\varepsilon \)-contaminated priors.” Annals of Statistics 14, 461-486. · Zbl 0602.62004
[8] Berger, J. O. and Wolpert, R. L. (1988). The Likelihood Principle. In IMS Lecture Notes, Ed. Gupta, S. S., Institute of Mathematical Statistics, vol. 6. · Zbl 1060.62500
[9] Birnbaum, Z. W. (1948). “On random variables with comparable peakedness.” Annals of Mathematical Statistics 19 76-81. · Zbl 0031.36801
[10] Christensen, R., Johnson, W. O., Branscum, A. J. and Hanson, T. E. (2011). Bayesian Ideas and Data Analysis. Boca Raton: CRC Press. · Zbl 1304.62006
[11] Cheney, W. (2001). Analysis for Applied Mathematics. New York: Springer. · Zbl 0984.46006
[12] de Carvalho, M., Page, G. L., and Barney, B. J. (2018). “Supplementary Material to “On the Geometry of Bayesian Inference”.” Bayesian Analysis.
[13] Diaconis, P. and Ylvisaker, D. (1979). “Conjugate priors for exponential families,” Annals of Statistics 7 269-281. · Zbl 0405.62011
[14] Evans, M. and Jang, G. H. (2011). “Weak informativity and the information in one prior relative to another.” Statistical Science 26, 423-439. · Zbl 1246.62007
[15] Evans, M. and Moshonov, H. (2006). “Checking for prior-data conflict.” Bayesian Analysis 1, 893-914. · Zbl 1331.62030
[16] Gelman, A., Jakulin, A., Pittau, M. G. and Su, Y. S. (2008). “A weakly informative default prior distribution for logistic and other regression models.” Annals of Applied Statistics 2, 1360-1383. · Zbl 1156.62017
[17] Gutiérrez-Peña, E. and Smith, A. F. M. (1995). “Conjugate parametrizations for natural exponential families.” Journal of the American Statistical Association 90, 1347-1356. · Zbl 0868.62029
[18] Giné, E. and Nickl, R. (2008). “A simple adaptive estimator of the integrated square of a density.” Bernoulli 14, 47-61. · Zbl 1155.62025
[19] Hartigan, J. A. (1998). “The maximum likelihood prior.” Annals of Statistics 26 2083-2103. · Zbl 0927.62023
[20] Hastie, T., Tibshirani, R. and Friedman, J. (2008). Elements of Statistical Learning. New York: Springer. · Zbl 0973.62007
[21] Hoff, P. (2009). A First Course in Bayesian Statistical Methods. New York: Springer. · Zbl 1213.62044
[22] Hunter, J. and Nachtergaele, B. (2005). Applied Analysis. London: World Scientific Publishing. · Zbl 0981.46002
[23] Kurtek, S. and Bharath, K. (2015). “Bayesian sensitivity analysis with the Fisher-Rao metric.” Biometrika 102, 601-616. · Zbl 1452.62252
[24] Kyung, M., Gill, J., Ghosh, M. and Casella, G. (2010). “Penalized regression, standard errors and Bayesian lassos.” Bayesian Analysis 5, 369-412. · Zbl 1330.62289
[25] Lavine, M. (1991). “Sensitivity in Bayesian statistics: The prior and the likelihood.” Journal of the American Statistical Association 86 396-399. · Zbl 0734.62005
[26] Lopes, H. F. and Tobias, J. L. (2011). “Confronting prior convictions: On issues of prior sensitivity and likelihood robustness in Bayesian analysis.” Annual Review of Economics 3, 107-131.
[27] Marriott, P. (2002). “On the local geometry of mixture models.” Biometrika 89 77-93. · Zbl 0998.62002
[28] Millman, R. S. and Parker, G. D. (1991). Geometry: A Metric Approach with Models. New York: Springer. · Zbl 0724.51001
[29] Newton, M. A. and Raftery, A. E. (1994). “Approximate Bayesian inference with the weighted likelihood Bootstrap (With Discussion).” Journal of the Royal Statistical Society, Series B, 56, 3-26. · Zbl 0788.62026
[30] Park, T. and Casella, G. (2008). “The Bayesian lasso.” Journal of the American Statistical Association 103, 681-686. · Zbl 1330.62292
[31] Raftery, A. E., Newton, M. A., Satagopan, J. M. and Krivitsky, P. N. (2007). “Estimating the integrated likelihood via posterior simulation using the harmonic mean identity.” In Bayesian Statistics, Eds. Bernardo, J. M., Bayarri, M. J., Berger, J. O., Dawid, A. P., Heckerman, D., Smith, A. F. M. and West, M., Oxford University Press, vol. 8. · Zbl 1252.62038
[32] Roos, M. and Held, L. (2011). “Sensitivity analysis for Bayesian hierarchical models.” Bayesian Analysis 6, 259-278. · Zbl 1330.62150
[33] Roos, M., Martins T. G., Held, L. and Rue, H. (2015). “Sensitivity analysis for Bayesian hierarchical models.” Bayesian Analysis 10, 321-349. · Zbl 1335.62059
[34] Slobodchikoff, C. N. and Schulz, W. C. (1980). “Measures of niche overlap.” Ecology 61 1051-1055.
[35] Scheel, I., Green, P. J. and Rougier, J. C. (2011). “A graphical diagnostic for identifying influential model choices in Bayesian hierarchical models.” Scandinavian Journal of Statistics 38, 529-550. · Zbl 1246.62064
[36] Shortle, J. F. and Mendel, M. B. (1996). “The geometry of Bayesian inference.” In Bayesian Statistics. eds. Bernardo, J. M., Berger, J. O., Dawid, A. P. and Smith, A. F. M., Oxford University Press, vol. 5, pp. 739-746.
[37] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge: Cambridge University Press. · Zbl 0910.62001
[38] Walter, G. and Augustin, T. (2009). “Imprecision and prior-data conflict in generalized Bayesian inference.” Journal of Statistical Theory and Practice 3, 255-271. · Zbl 1211.62051
[39] Wolpert, R. and Schmidler, S. (2012). “\( \alpha \)-stable limit laws for harmonic mean estimators of marginal likelihoods.” Statistica Sinica 22, 655-679. · Zbl 06072105
[40] Zhu, H., Ibrahim, J. G. and Tang, N. (2011). “Bayesian influence analysis: A geometric approach.” Biometrika 98, 307-323. · Zbl 1215.62029
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.