Bayesian design of experiments for intractable likelihood models using coupled auxiliary models and multivariate emulation. (English) Zbl 1437.62303

Summary: A Bayesian design is given by maximising an expected utility over a design space. The utility is chosen to represent the aim of the experiment and its expectation is taken with respect to all unknowns: responses, parameters and/or models. Although straightforward in principle, there are several challenges to finding Bayesian designs in practice. Firstly, the utility and expected utility are rarely available in closed form and require approximation. Secondly, the design space can be of high-dimensionality. In the case of intractable likelihood models, these problems are compounded by the fact that the likelihood function, whose evaluation is required to approximate the expected utility, is not available in closed form. A strategy is proposed to find Bayesian designs for intractable likelihood models. It relies on the development of an automatic, auxiliary modelling approach, using multivariate Gaussian process emulators, to approximate the likelihood function. This is then combined with a copula-based approach to approximate the marginal likelihood (a quantity commonly required to evaluate many utility functions). These approximations are demonstrated on examples of stochastic process models involving experimental aims of both parameter estimation and model comparison.


62K05 Optimal statistical designs
62F15 Bayesian inference
62P10 Applications of statistics to biology and medical sciences; meta analysis
62H05 Characterization and structure theory for multivariate probability distributions; copulas
60G15 Gaussian processes


acebayes; BayesDA
Full Text: DOI arXiv Euclid


[1] Abramowitz, M. and Stegun, I. A. (2002). “Handbook of Mathematical Functions.” Available at URL www.math.sfu.ca/ cbm/aands/toc.htm. · Zbl 0171.38503
[2] Amzal, B., Bois, F. Y., Parent, E., and Robert, C. P. (2006). “Bayesian-optimal design via interacting particle systems.” Journal of the American Statistical Association, 101: 773-785. · Zbl 1119.62308 · doi:10.1198/016214505000001159
[3] Atkinson, A. C., Donev, A. N., and Tobias, R. D. (2007). Optimum experimental designs, with SAS. Oxford University Press Inc., New York. · Zbl 1183.62129
[4] Chaloner, K. and Verdinelli, I. (1995). “Bayesian Experimental Design: A Review.” Statistical Science, 10: 273-304. · Zbl 0955.62617 · doi:10.1214/ss/1177009939
[5] Conti, S. and O’Hagan, A. (2010). “Bayesian Emulation of Complex Multi-Output and Dynamic Computer Models.” Journal of Statistical Planning and Inference, 140: 640-651. · Zbl 1177.62033 · doi:10.1016/j.jspi.2009.08.006
[6] Dean, A., Morris, M., Stufken, J., and Bingham, D. (eds.) (2015). Handbook of Design and Analysis of Experiments. Boca Raton: CRC Press. · Zbl 1327.62001
[7] Dehideniya, M., Drovandi, C. C., and McGree, J. M. (2018). “Optimal Bayesian Design for Discriminating between Models with Intractable Likelihoods in Epidemiology.” Computational Statistics and Data Analysis, 124: 277-297. · Zbl 1469.62053 · doi:10.1016/j.csda.2018.03.004
[8] Demarta, S. and McNeil, A. (2005). “The t Copula and Related Copulas.” International Statistical Review, 73(1): 111-129. · Zbl 1104.62060 · doi:10.1111/j.1751-5823.2005.tb00254.x
[9] Denham, D., Ponnudurai, T., Nelson, G., Guy, F., and Rogers, R. (1977). “Studies with Brugia pahangi. I. Parasitological observations on primary infections of cats (Felis catus).” Journal for Parasitology, 2: 239-247.
[10] Drovandi, C. C. and Pettitt, A. N. (2013). “Bayesian experimental design for models with intractable likelihoods.” Biometrics, 69(4): 937-948. · Zbl 1419.62182 · doi:10.1111/biom.12081
[11] Drovandi, C. C., Pettitt, A. N., and Faddy, M. J. (2011). “Approximate Bayesian computation using indirect inference.” Journal of the Royal Statistical Society: Series C (Applied Statistics), 60(3): 503-524.
[12] Drovandi, C. C., Pettitt, A. N., and Lee, A. (2015). “Bayesian Indirect Inference Using a Parametric Auxiliary Model.” Statistical Science, 30(1): 72-95. · Zbl 1332.62088 · doi:10.1214/14-STS498
[13] Gelman, A., Carlin, J., Stern, H., Dunson, D., Vehtari, A., and Rubin, D. (2014). Bayesian Data Analysis. Chapman and Hall, 3rd edition. · Zbl 1279.62004
[14] Gillespie, C. and Boys, R. (2019). “Efficient construction of Bayes optimal designs for stochastic process models.” Statistics and Computing, In press. · Zbl 1430.62175 · doi:10.1007/s11222-018-9833-3
[15] Gillespie, C. and Golightly, A. (2010). “Bayesian inference for generalized stochastic population growth models with application to aphids.” Journal of Royal Statistical Society, Series C, 59(2): 341-357.
[16] Gillespie, D. (1977). “Exact stochastic inference of coupled chemical reactions.” Journal of Physical Chemistry, 81: 2340-2361.
[17] Gourieroux, C., Monfort, A., and Renault, E. (1993). “Indirect Inference.” Journal of Applied Econometrics, 8: 85-118. · Zbl 1448.62202 · doi:10.1007/BF02589066
[18] Hainy, M., Müller, W., and Wagner, H. (2013). “Likelihood-Free Simulation-Based Optimal Design: An Introduction.” In Melas, V., Mignani, S., Monari, P., and Salmaso, L. (eds.), Topics in Statistical Simulation, 271-278.
[19] Heggland, K. and Frigessi, A. (2004). “Estimating functions in indirect inference.” Journal of the Royal Statistical Society Series B, 104: 117-131. · Zbl 1062.62098 · doi:10.1111/j.1369-7412.2003.05341.x
[20] Huan, X. and Marzouk, Y. M. (2013). “Simulation-Based Optimal Bayesian Experimental Design for Nonlinear Systems.” Journal of Computational Physics, 232(1): 288-317.
[21] Joe, H. (1997). Multivariate models and dependence concepts. Chapman & Hall. · Zbl 0990.62517
[22] Jones, M., Goldstein, M., Jonathan, P., and Randell, D. (2016). “Bayes linear analysis for Bayesian optimal experimental design.” Journal of Statistical Planning and Inference, 171: 115-129. · Zbl 1336.62209 · doi:10.1016/j.jspi.2015.10.011
[23] Lange, K. (2013). Optimization. Springer, 2nd edition.
[24] Lindley, D. (1956). “On a measure of the information provided by an experiment.” Annals of Mathematical Statistics, 27: 986-1005. · Zbl 0073.14103 · doi:10.1214/aoms/1177728069
[25] Long, Q., Scavino, M., Tempone, R., and Wang, S. (2013). “Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations.” Computer Methods in Applied Mechanics and Engineering, 259: 24-39. · Zbl 1286.62068 · doi:10.1016/j.cma.2013.02.017
[26] Matis, J., Kiffe, T., Matis, T., and Stevenson, D. (2007). “Stochastic modeling of aphid population growth with nonlinear, power-law dynamics.” Mathematical Biosciences, 208(2): 469-494. · Zbl 1119.92052 · doi:10.1016/j.mbs.2006.11.004
[27] Meyer, R. and Nachtsheim, C. (1995). “The coordinate-exchange algorithm for constructing exact optimal experimental designs.” Technometerics, 37(1): 60-69. · Zbl 0825.62652 · doi:10.1080/00401706.1995.10485889
[28] Müller, P. (1999). “Simulation-Based Optimal Design.” Bayesian Statistics, 6: 459-474. · Zbl 0974.62058
[29] Müller, P. and Parmigiani, G. (1995). “Optimal Design via Curve Fitting of Monte Carlo Experiments.” Journal of the American Statistical Association, 90(432): 1322-1330. · Zbl 0867.62065
[30] Müller, P., Sanso, B., and De Iorio, M. (2004). “Optimal Bayesian design by inhomogeneous Markov chain simulation.” Journal of the American Statistical Association, 99: 788-798. · Zbl 1117.62404 · doi:10.1198/016214504000001123
[31] Nelson, R. (1998). An Introduction to Copulas. Springer Verlag New York.
[32] Overstall, A. and McGree, J. (2019). “Supplementary Material for “Bayesian Design of Experiments for Intractable Likelihood Models Using Coupled Auxiliary Models and Multivariate Emulation”.” Bayesian Analysis. · Zbl 1437.62303
[33] Overstall, A. M., McGree, J. M., and Drovandi, C. C. (2018a). “An approach for finding fully Bayesian optimal designs using normal-based approximations to loss functions.” Statistics and Computing, 28(2): 343-358. · Zbl 1384.62277 · doi:10.1007/s11222-017-9734-x
[34] Overstall, A. M. and Woods, D. C. (2017). “Bayesian Design of Experiments using Approximate Coordinate Exchange.” Technometrics, 59(4): 458-470. · doi:10.1080/00401706.2016.1251495
[35] Overstall, A. M., Woods, D. C., and Adamou, M. (2018b). acebayes: Optimal Bayesian Experimental Design using the ACE Algorithm. R package version 1.6.0. URL https://cran.r-project.org/web/packages/acebayes/index.html
[36] Pagendam, D. and Pollett, P. (2013). “Optimal design of experimental epidemics.” Journal of Statistical Planning and Inference, 143(3): 563-572. · Zbl 1428.62477 · doi:10.1016/j.jspi.2012.09.011
[37] Panagiotelis, A., Czado, C., and Joe, H. (2012). “Pair Copula Constructions for Multivariate Discrete Data.” Journal of American Statistical Association, 107: 1063-1072. · Zbl 1395.62114 · doi:10.1080/01621459.2012.682850
[38] Parker, B., Gilmour, S., Schormans, J., and Maruri-Aguilar, H. (2015). “Optimal design of measurements on queueing systems.” Queueing Systems, 79: 365-390. · Zbl 1309.90018 · doi:10.1007/s11134-014-9421-y
[39] Price, D., Bean, N., Ross, J., and Tuke, J. (2016). “On the efficient determination of optimal Bayesian experimental designs using ABC: A case study in optimal observation of epidemics.” Journal of Statistical Planning and Inference, 172: 1-15. · Zbl 1334.62198 · doi:10.1016/j.jspi.2015.12.008
[40] Qian, P., Wu, H., and Wu, C. (2008). “Gaussian Process Models for Computer Experiments with Qualitative and Quantitative Factors.” Technometrics, 50: 383-396. · doi:10.1198/004017008000000262
[41] Riley, S., Donnelly, C., and Ferguson, N. (2003). “Robust parameter estimation techniques for stochastic within-host macroparasite models.” Journal of Theoretical Biology, 419-430. · Zbl 1465.92127
[42] Ryan, C., Drovandi, C., and Pettitt, A. N. (2016a). “Optimal Bayesian Experimental Design for Models with Intractable Likelihoods Using Indirect Inference Applied to Biological Process Models.” Bayesian Analysis, 11(3): 857-883. · Zbl 1359.62321 · doi:10.1214/15-BA977
[43] Ryan, E. G., Drovandi, C. C., McGree, J. M., and Pettitt, A. N. (2016b). “A Review of Modern Computational Algorithms for Bayesian Optimal Design.” International Statistical Review, 84: 128-154. · Zbl 07763475
[44] Ryan, E. G., Drovandi, C. C., Thompson, M. H., and Pettitt, A. N. (2014). “Towards Bayesian experimental design for nonlinear models that require a large number of sampling times.” Computational Statistics and Data Analysis, 70: 45-60. · Zbl 1471.62175 · doi:10.1016/j.csda.2013.08.017
[45] Ryan, K. (2003). “Estimating expected information gains for experimental designs with application to the random fatigue-limit model.” Journal of Computational and Graphical Statistics, 12: 585-603.
[46] Santner, T., Williams, B., and Notz, W. (2003). The Design and Analysis of Computer Experiments. Springer, New York. · Zbl 1041.62068
[47] Tavaré, S., Balding, D. J., Griffiths, R., and Donnelly, P. (1997). “Inferring Coalescence Times from DNA Sequence Data.” Genetics, 145(2): 505-518.
[48] Tran, M., Nott, D. J., and Kohn, R. (2017). “Variational Bayes with Intractable Likelihood.” Journal of Computational and Graphical Statistics, To appear.
[49] Weaver, B., Williams, B., Anderson-Cook, C., and Higdon, D. (2016). “Computational Enhancements to Bayesian Design of Experiments Using Gaussian Processes.” Bayesian Analysis, 11(1): 191-213. · Zbl 1359.62322 · doi:10.1214/15-BA945
[50] Wood, S. (2010). “Statistical inference for noisy nonlinear ecological dynamic systems.” Nature, 466: 1102-1104.
[51] Woods, D., Overstall, A., Adamou, M., and Waite, T. (2017). “Bayesian design of experiments for generalised linear models and dimensional analysis with industrial and scientific application.” Quality Engineering, 29(1): 91-103.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.