# zbMATH — the first resource for mathematics

Posterior belief assessment: extracting meaningful subjective judgements from Bayesian analyses with complex statistical models. (English) Zbl 1335.62024
Summary: We are concerned with attributing meaning to the results of a Bayesian analysis for a problem which is sufficiently complex that we are unable to assert a precise correspondence between the expert probabilistic judgements of the analyst and the particular forms chosen for the prior specification and the likelihood for the analysis. In order to do this, we propose performing a finite collection of additional Bayesian analyses under alternative collections of prior and likelihood modelling judgements that we may also view as representative of our prior knowledge and the problem structure, and use these to compute posterior belief assessments for key quantities of interest. We show that these assessments are closer to our true underlying beliefs than the original Bayesian analysis and use the temporal sure preference principle to establish a probabilistic relationship between our true posterior judgements, our posterior belief assessment and our original Bayesian analysis to make this precise. We exploit second order exchangeability in order to generalise our approach to situations where there are infinitely many alternative Bayesian analyses we might consider as informative for our true judgements so that the method remains tractable even in these cases. We argue that posterior belief assessment is a tractable and powerful alternative to robust Bayesian analysis. We describe a methodology for computing posterior belief assessments in even the most complex of statistical models and illustrate with an example of calibrating an expensive ocean model in order to quantify uncertainty about global mean temperature in the real ocean.

##### MSC:
 62C10 Bayesian problems; characterization of Bayes procedures 65C60 Computational problems in statistics (MSC2010) 65C05 Monte Carlo methods
##### Software:
MATCH Uncertainty Elicitation; BayesDA
Full Text:
##### References:
 [1] Andiranakis, I. and Challenor, P. G. (2012), The effect of the nugget on Gaussian process emulators of computer models. Computational Statistics and Data Analysis , 56: 4215-4228. · Zbl 1255.62306 [2] Bayarri, M. J., Berger, J. O., Cafeo, J., Garcia-Donato, G., Liu, F., Palomo, J., Parthasarathy, R. J., Paulo, R., Sacks, J., and Walsh, D. (2007), Computer model validation with functional output, The Annals of Statistics , 35, 1874-1906. · Zbl 1144.62368 [3] Berger, J. O. (1994), An overview of robust Bayesian analysis, Test , 3(1), 5-124. · Zbl 0827.62026 [4] Berger, J. O. (2006), The case for objective Bayesian analysis. Bayesian Analysis , 1(3), 385-402. · Zbl 1331.62042 [5] Craig, P. S., Goldstein, M., Rougier, J. C., and Seheult, A. H. (2001), Bayesian forecasting for complex systems using computer simulators, Journal of the American Statistical Association , 96, 717-729. · Zbl 1017.62019 [6] Cumming, J. A. and Goldstein, M. (2009), Small sample designs for complex high-dimensional models based on fast approximations, Technometrics , 51, 377-388. [7] de Finetti, B. (1974), Theory of Probability, Volume 1 , Wiley, New York. · Zbl 0694.60001 [8] de Finetti, B. (1975), Theory of Probability, Volume 2 , Wiley, New York. · Zbl 0694.60001 [9] Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2004), Bayesian Data Analysis, second edition , Chapman and Hall/CRC Texts in Statistical Science. · Zbl 1039.62018 [10] Goldstein, M. (1997). Prior inferences for posterior judgements. In: Structures and Norms in Science , M. C. D. Chiara et al., eds., Kluwer, 55-71. · Zbl 0987.03019 [11] Goldstein, M. (2006), Subjective Bayesian analysis: Principles and practice. Bayesian Analysis , 1(3), 403-420. · Zbl 1331.62047 [12] Goldstein, M. (2011). External Bayesian analysis for computer simulators. In: Bayesian Statistics 9 . Bernado, J. M. et al., eds., Oxford University Press. [13] Goldstein, M. (2012). Observables and models: exchangeability and the inductive argument. In: Bayesian Theory and Applications , Damien, P. et al., eds., Clarendon Press, Oxford, 3-18. [14] Goldstein, M. and Wooff, D. (2007), Bayes Linear Statistics Theory and Methods , John Wiley and Sons Ltd. · Zbl 1231.62033 [15] Haylock, R. and O’Hagan, A. (1996), “On inference for outputs of computationally expensive algorithms with uncertainty on the inputs.” In: Bayesian Statistics 5 , Bernado, J. M., Berger, J. O., Dawid, A. P., and Smith, A. F. M., eds., Oxford University Press, 629-637. [16] Higdon, D., Nakhleh, C., Gattiker, J., and Williams, B. (2008), “A Bayesian calibration approach to the thermal problem”, Computer Methods in Applied Mechanics and Engineering, 197, 2431-2441. · Zbl 1388.80005 [17] Hoeting, J. A., Madigan, D., Raftery, A. E., and Volinsky, V. T. (1999), Bayesian model averaging: a tutorial, Statistical Science , 14(4), 382-401. · Zbl 1059.62525 [18] Ingleby, B. and Huddleston, M. (2007), Quality control of ocean temperature and salinity profiles - Historical and real time data, Journal of Marine Systems , 65 158-175. [19] Kaufman, C. G., Bingham, D., Habib, S., Heitmann, K., and Frieman, J. A. (2011), Efficient emulators of computer experiments using compactly supported correlation functions, with an application to cosmology, The Annals of Applied Statistics, 5(4), 2470-2492. · Zbl 1234.62166 [20] Kennedy, M. C. and O’Hagan, A. (2001), Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63: 425-464. · Zbl 1007.62021 [21] Lee, L., Pringle, K., Reddington, C., Mann, G., Stier, P., Spracklen, D., Pierce, J., and Carslaw, K. (2013), The magnitude and causes of uncertainty in global model simulations of cloud condensation nuclei, Atmospheric Chemistry and Physics , 13.17, 8879-8914. [22] Liang, F., Liu, C., and Carroll, R. J. (2010), Advanced Markov Chain Monte Carlo Methods , John Wiley and Sons Ltd., Chichester, UK. · Zbl 1209.62009 [23] Lindley, D. V. (2000), The philosophy of statistics. Journal of the Royal Statistical Society: Series D (The Statistician), 49(3), 293-337. [24] Madec, G. (2008), NEMO ocean engine. Note du Pole de Modélisation . Institut Pierre-Simon Laplace (IPSL), France, No. 27, ISSN No. 1288-1619. [25] Morris, D. E., Oakley, J. E., and Crowe, J. A. (2014), A web-based tool for eliciting probability distributions from experts. Environmental Modelling and Software , 52: 1-4, ISSN 1364-8152, [26] Oakley, J. E. and O’Hagan, A. (2010). SHELF: the Sheffield Elicitation Framework (Version 2.0) , School of Mathematics and Statistics, University of Sheffield (2010), [27] Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P. (1989), Design and analysis of computer experiments, Statistical Science , 4, 409-435. · Zbl 0955.62619 [28] Santner, T. J., Williams, B. J., and Notz, W. I. (2003), The Design and Analysis of Computer Experiments , Springer-Verlag New York. · Zbl 1041.62068 [29] Savage, L. J. (1977) The shifting foundations of statistics. In: Logic, Laws and Life: Some Philosophical Complications , R.G. Colodny, ed., Pittsburgh University Press, Pittsburg, 3-18. [30] Sexton, D. M. H., Murphy, J. M., and Collins, M. (2011), Multivariate probabilistic projections using imperfect climate models part 1: outline of methodology, Climate Dynamics , [31] Vernon, I., Goldstein, M., and Bower, R. G. (2010), “Galaxy formation: a Bayesian uncertainty analysis,” Bayesian Analysis, 5(4), 619-846, with discussion. · Zbl 1330.85005 [32] Williamson, D. and Blaker, A. T. (2014), Evolving Bayesian emulators for structurally chaotic time series with application to large climate models. SIAM/ASA Journal on Uncertainty Quantification , 2(1) 1-28. · Zbl 1349.62433 [33] Williamson, D., Goldstein, M., and Blaker, A. (2012), “Fast linked analyses for scenario based hierarchies,” Journal of the Royal Statistical Society: Series C (Applied Statistics) , 61(5), 665-692. [34] Williamson, D., Goldstein, M., Allison, L., Blaker, A., Challenor, P. Jackson, L., and Yamazaki, K. (2013), History matching for exploring and reducing climate model parameter space using observations and a large perturbed physics ensemble. Climate Dynamics 41: 1703-1729. [35] Williamson, D. (2015). Exploratory ensemble designs for environmental models using $$k$$-extended Latin hypercubes Environmetrics , 26(4) 268-283. [36] Williamson, D., Blaker, A. T., Hampton, C., and Salter, J. (2014). Identifying and removing structural biases in climate models with history matching. Climate Dynamics , Online First, [37] Williamson, D., Blaker, A. T., and Sinha, B. (2015) Statistical ocean model tuning and parametric uncertainty quantification with NEMO, In preparation .
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.