zbMATH — the first resource for mathematics

Approximate Bayesian model inversion for PDEs with heterogeneous and state-dependent coefficients. (English) Zbl 1453.62410
Summary: We present two approximate Bayesian inference methods for parameter estimation in partial differential equation (PDE) models with space-dependent and state-dependent parameters. We demonstrate that these methods provide accurate and cost-effective alternatives to Markov Chain Monte Carlo simulation. We assume a parameterized Gaussian prior on the unknown functions, and approximate the posterior density by a parameterized multivariate Gaussian density. The parameters of the prior and posterior are estimated from sparse observations of the PDE model’s states and the unknown functions themselves by maximizing the evidence lower bound (ELBO), a lower bound on the log marginal likelihood of the observations. The first method, Laplace-EM, employs the expectation maximization algorithm to maximize the ELBO, with a Laplace approximation of the posterior on the E-step, and minimization of a Kullback-Leibler divergence on the M-step. The second method, DSVI-EB, employs the doubly stochastic variational inference (DSVI) algorithm, in which the ELBO is maximized via gradient-based stochastic optimization, with noisy gradients computed via simple Monte Carlo sampling and Gaussian backpropagation. We apply these methods to identifying diffusion coefficients in linear and nonlinear diffusion equations, and we find that both methods provide accurate estimates of posterior densities and the hyperparameters of Gaussian priors. While the Laplace-EM method is more accurate, it requires computing Hessians of the physics model. The DSVI-EB method is found to be less accurate but only requires gradients of the physics model.
62F15 Bayesian inference
35B30 Dependence of solutions to PDEs on initial and/or boundary data and/or on parameters of PDEs
ADVI; iLaplace; EnKF
Full Text: DOI
[1] Stuart, A. M., Inverse problems: a Bayesian perspective, Acta Numer., 19, 451-559 (2010) · Zbl 1242.65142
[2] Hanke, M., A regularizing Levenberg-Marquardt scheme, with applications to inverse groundwater filtration problems, Inverse Probl., 13, 1, 79 (1997) · Zbl 0873.65057
[3] Barajas-Solano, D. A.; Wohlberg, B. E.; Vesselinov, V. V.; Tartakovsky, D. M., Linear functional minimization for inverse modeling, Water Resour. Res., 51, 4516-4531 (2014)
[4] Evensen, G., Data Assimilation: The Ensemble Kalman Filter (2006), Springer-Verlag: Springer-Verlag Berlin, Heidelberg · Zbl 1157.86001
[5] Salimans, T.; Kingma, D.; Welling, M., Markov chain Monte Carlo and variational inference: bridging the gap, (Bach, F.; Blei, D., Proceedings of the 32nd International Conference on Machine Learning. Proceedings of the 32nd International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 37 (2015), PMLR: PMLR Lille, France), 1218-1226
[6] Goodman, J.; Weare, J., Ensemble samplers with affine invariance, Commun. Appl. Math. Comput. Sci., 5, 1, 65-80 (2010) · Zbl 1189.65014
[7] Neiswanger, W.; Wang, C.; Xing, E., Asymptotically exact, embarrassingly parallel MCMC, ArXiv e-prints
[8] Hoffman, M. D.; Gelman, A., The no-u-turn sampler: adaptively setting path lengths in hamiltonian Monte Carlo, J. Mach. Learn. Res., 15, 1593-1623 (2014) · Zbl 1319.60150
[9] Rasmussen, C. E.; Williams, C. K.I., Gaussian Processes for Machine Learning, Adaptive Computation and Machine Learning (2005), The MIT Press
[10] Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Numerical gaussian processes for time-dependent and nonlinear partial differential equations, SIAM J. Sci. Comput., 40, 1, A172-A198 (2018) · Zbl 1386.65030
[11] Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Machine learning of linear differential equations using gaussian processes, J. Comput. Phys., 348, 683-693 (2017) · Zbl 1380.68339
[12] Bishop, C. M., Pattern Recognition and Machine Learning, Information Science and Statistics (2006), Springer-Verlag: Springer-Verlag Berlin, Heidelberg · Zbl 1107.68072
[13] Lawrence, N. D.; Sanguinetti, G.; Rattray, M., Modelling transcriptional regulation using gaussian processes, (Schölkopf, B.; Platt, J. C.; Hoffman, T., Advances in Neural Information Processing Systems, vol. 19 (2007), MIT Press), 785-792
[14] Neal, R. M.; Hinton, G. E., A View of the EM Algorithm that Justifies Incremental, Sparse, and Other Variants, 355-368 (1998), Springer: Springer Netherlands, Dordrecht · Zbl 0916.62019
[15] Ranganath, R.; Gerrish, S.; Blei, D., Black box variational inference, (Kaski, S.; Corander, J., Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics. Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol. 33 (2014), PMLR: PMLR Reykjavik, Iceland), 814-822
[16] Titsias, M.; Lázaro-Gredilla, M., Doubly stochastic variational Bayes for non-conjugate inference, (Xing, E. P.; Jebara, T., Proceedings of the 31st International Conference on Machine Learning. Proceedings of the 31st International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 32 (2014), PMLR: PMLR Beijing, China), 1971-1979
[17] Minka, T. P., Expectation propagation for approximate Bayesian inference, (Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence. Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, UAI’01 (2001), Morgan Kaufmann Publishers Inc.: Morgan Kaufmann Publishers Inc. San Francisco, CA, USA), 362-369
[18] Tsilifis, P.; Bilionis, I.; Katsounaros, I.; Zabaras, N., Computationally efficient variational approximations for Bayesian inverse problems, J. Verif. Valid. Uncertain. Quantificat., 1, 3, Article 031004 pp. (2016)
[19] Jin, B.; Zou, J., Hierarchical Bayesian inference for ill-posed problems via variational method, J. Comput. Phys., 229, 19, 7317-7343 (2010) · Zbl 1198.65189
[20] Franck, I. M.; Koutsourelakis, P., Sparse variational Bayesian approximations for nonlinear inverse problems: applications in nonlinear elastography, Comput. Methods Appl. Mech. Eng., 299, 215-244 (2016) · Zbl 1425.65132
[21] Guha, N.; Wu, X.; Efendiev, Y.; Jin, B.; Mallick, B. K., A variational Bayesian approach for inverse problems with skew-t error distributions, J. Comput. Phys., 301, 377-393 (2015) · Zbl 1349.62079
[22] Yang, K.; Guha, N.; Efendiev, Y.; Mallick, B. K., Bayesian and variational Bayesian approaches for flows in heterogeneous random media, J. Comput. Phys., 345, 275-293 (2017) · Zbl 1378.76116
[23] Blei, D. M.; Kucukelbir, A.; McAuliffe, J. D., Variational inference: a review for statisticians, J. Am. Stat. Assoc., 112, 518, 859-877 (2017)
[24] Kucukelbir, A.; Tran, D.; Ranganath, R.; Gelman, A.; Blei, D. M., Automatic differentiation variational inference, J. Mach. Learn. Res., 18, 14, 1-45 (2017) · Zbl 1437.62109
[25] Kingma, D. P.; Welling, M., Auto-encoding variational Bayes, ArXiv e-prints
[26] Rezende, D. J.; Mohamed, S.; Wierstra, D., Stochastic backpropagation and approximate inference in deep generative models, ArXiv e-prints
[27] Challis, E.; Barber, D., Gaussian Kullback-Leibler approximate inference, J. Mach. Learn. Res., 14, 2239-2286 (2013) · Zbl 1318.62085
[28] Williams, R. J., Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning, 5-32 (1992), Springer: Springer US, Boston, MA · Zbl 0772.68076
[29] Friston, K.; Mattout, J.; Trujillo-Barreto, N.; Ashburner, J.; Penny, W., Variational free energy and the Laplace approximation, NeuroImage, 34, 1, 220-234 (2007)
[30] Ruli, E.; Sartori, N.; Ventura, L., Improved Laplace approximation for marginal likelihoods, Electron. J. Stat., 10, 2, 3986-4009 (2016) · Zbl 1357.62129
[31] Giles, M. B.; Duta, M. C.; Müller, J.-D.; Pierce, N. A., Algorithm developments for discrete adjoint methods, AIAA J., 41, 2, 198-205 (2003)
[32] Ghate, D.; Giles, M., Efficient Hessian calculation using automatic differentiation, (25th AIAA Applied Aerodynamics Conference (2007)), 4059
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.