×

A Laplace method for under-determined Bayesian optimal experimental designs. (English) Zbl 1426.62221

Summary: In [the authors, ibid. 259, 24–39 (2013; Zbl 1286.62068)], a new method based on the Laplace approximation was developed to accelerate the estimation of the post-experimental expected information gains (Kullback-Leibler divergence) in model parameters and predictive quantities of interest in the Bayesian framework. A closed-form asymptotic approximation of the inner integral and the order of the corresponding dominant error term were obtained in the cases where the parameters are determined by the experiment. In this work, we extend that method to the general case where the model parameters cannot be determined completely by the data from the proposed experiments. We carry out the Laplace approximations in the directions orthogonal to the null space of the Jacobian matrix of the data model with respect to the parameters, so that the information gain can be reduced to an integration against the marginal density of the transformed parameters that are not determined by the experiments. Furthermore, the expected information gain can be approximated by an integration over the prior, where the integrand is a function of the posterior covariance matrix projected over the aforementioned orthogonal directions. To deal with the issue of dimensionality in a complex problem, we use either Monte Carlo sampling or sparse quadratures for the integration over the prior probability density function, depending on the regularity of the integrand function. We demonstrate the accuracy, efficiency and robustness of the proposed method via several nonlinear under-determined test cases. They include the designs of the scalar parameter in a one dimensional cubic polynomial function with two unidentifiable parameters forming a linear manifold, and the boundary source locations for impedance tomography in a square domain, where the unknown parameter is the conductivity, which is represented as a random field.

MSC:

62K05 Optimal statistical designs
62-08 Computational methods for problems pertaining to statistics
62F15 Bayesian inference
62B10 Statistical aspects of information-theoretic topics
65N21 Numerical methods for inverse problems for boundary value problems involving PDEs

Citations:

Zbl 1286.62068
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[1] Ginebra, J., On the measure of the information in a statistical experiment, Bayesian Anal., 2, 167-211 (2007) · Zbl 1331.62056
[2] Chaloner, K.; Verdinelli, I., Bayesian experimental design: a review, Statist. Sci., 10, 273-304 (1995) · Zbl 0955.62617
[3] Stigler, S. M., Laplace’s 1774 memoir on inverse probability, Statist. Sci., 1, 359-363 (1986) · Zbl 0955.01503
[4] Tierney, L.; Kass, R. E.; Kadane, J. B., Fully exponential Laplace approximations to expectations and variances of nonpositive functions, J. Amer. Statist. Assoc., 84, 710-716 (1989) · Zbl 0682.62012
[5] Kass, R. E.; Tierney, L.; Kadane, J. B., The validity of posterior expansions based on Laplace’s method, (Geisser, S.; Hodges, J. S.; Press, S. J.; Zellner, A., Essays in Honor of George Barnard (1990), North-Holland: North-Holland Amsterdam), 473-488 · Zbl 0734.62034
[6] Kass, R. E.; Tierney, L.; Kadane, J. B., Laplace’s method in Bayesian analysis, Contemp. Math., 115, 89-99 (1991) · Zbl 0733.65108
[7] Long, Q.; Scavino, M.; Tempone, R.; Wang, S., Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations, Comput. Methods Appl. Mech. Engrg., 259, 24-39 (2013) · Zbl 1286.62068
[8] Clarke, B. S.; Barron, A. R., Entropy Risk and the Bayesian Central Limit Theorem, Technical Report 91-56 (1991), Department of Statistics, Purdue University
[9] Polson, N. G., On the expected amount of information from a non-linear model, J. R. Stat. Soc. Ser. B Stat. Methodol., 54, 889-895 (1992) · Zbl 0774.62064
[10] Ghosal, S.; Samanta, T., Expansion of Bayes risk for entropy loss and reference prior in nonregular cases, Statist. Decisions, 15, 129-140 (1997) · Zbl 0902.62008
[11] Bernardo, J. M., Reference posterior distributions for Bayesian inference, J. R. Stat. Soc. Ser. B Stat. Methodol., 41, 113-147 (1979) · Zbl 0428.62004
[12] Polson, N. G., Bayesian perspectives on statistical modelling (1988), Department of Mathematics, University of Nottingham, (Ph.D. thesis)
[13] Clarke, B. S.; Wasserman, L., Noninformative priors and nuisance parameters, J. Amer. Statist. Assoc., 88, 1427-1432 (1993) · Zbl 0793.62016
[14] Clarke, B. S.; Yuan, A., Partial information reference priors: derivation and interpretations, J. Statist. Plann. Inference, 123, 313-345 (2004) · Zbl 1053.62010
[15] Joannides, M.; Le Gland, F., Small noise asymptotics of the Bayesian estimator in nonidentifiable models, Stat. Inference Stoch. Process., 5, 95-130 (2002) · Zbl 1006.62014
[16] Hwang, C.-R., Laplace’s method revisited: weak convergence of probability measures, Ann. Probab., 8, 1177-1182 (1980) · Zbl 0452.60007
[17] Spivak, M., A Comprehensive Introduction to Differential Geometry, vol. I (1999), Publish or Perish, Inc.: Publish or Perish, Inc. Houston, Texas · Zbl 1213.53001
[18] Hager, W. W., Updating the inverse of a matrix, SIAM Rev., 31, 221-239 (1989) · Zbl 0671.65018
[19] Smolyak, S. A., Quadrature and interpolation formulas for tensor products of certain classes of functions, Soviet Math. Dokl., 4, 240-243 (1963) · Zbl 0202.39901
[20] Barthelmann, V.; Novak, E.; Ritter, K., High dimensional polynomial interpolation on sparse grids, Adv. Comput. Math., 12, 273-288 (2000) · Zbl 0944.41001
[21] Nobile, F.; Tempone, R.; Webster, C. G., A sparse grid stochastic collocation method for partial differential equations with random input data, SIAM J. Numer. Anal., 46, 2309-2345 (2008) · Zbl 1176.65137
[22] Somersalo, E.; Cheney, M.; Isaacson, D., Existence and uniqueness for electrode models for electric current computed tomography, SIAM J. Appl. Math, 52, 1023-1040 (1992) · Zbl 0759.35055
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.