Hierarchical Bayesian inference for ill-posed problems via variational method. (English) Zbl 1198.65189

Authors’ abstract: This paper investigates a novel approximate Bayesian inference procedure for numerically solving inverse problems. A hierarchical formulation which determines automatically the regularization parameter and the noise level together with the inverse solution is adopted. The framework is of variational type, and it can deliver the inverse solution and regularization parameter together with their uncertainties calibrated. It approximates the posteriori probability distribution by separable distributions based on Kullback-Leibler divergence.
Two approximations are derived within the framework, and some theoretical properties, e.g. variance estimate and consistency, are also provided. Algorithms for their efficient numerical realization are described, and their convergence properties are also discussed. Extensions to nonquadratic regularization/nonlinear forward models are also briefly studied. Numerical results for linear and nonlinear Cauchy-type problems arising in heat conduction with both smooth and nonsmooth solutions are presented for the proposed method, and compared with that by Markov chain Monte Carlo. The results illustrate that the variational method can faithfully capture the posteriori distribution in a computationally efficient way.


65M32 Numerical methods for inverse problems for initial value and initial-boundary value problems involving PDEs
62F15 Bayesian inference
65M60 Finite element, Rayleigh-Ritz and Galerkin methods for initial value and initial-boundary value problems involving PDEs
35K05 Heat equation
65M30 Numerical methods for ill-posed problems for initial value and initial-boundary value problems involving PDEs
35R25 Ill-posed problems for PDEs
35R30 Inverse problems for PDEs
65M12 Stability and convergence of numerical methods for initial value and initial-boundary value problems involving PDEs
Full Text: DOI


[1] Beck, J. V.; Blackwell, B.; St Clair, C. R., Inverse Heat Conduction: Ill-Posed Problems (1985), Wiley: Wiley New York · Zbl 0633.73120
[2] Tarantola, A., Inverse Problem Theory and Methods for Model Parameter Estimation (2005), SIAM: SIAM Philadelphia · Zbl 1074.65013
[3] Malinverno, A.; Briggs, V. A., Expanded uncertainty quantification in inverse problems: hierarchical Bayes and empirical Bayes, Geophysics, 69, 1005-1016 (2004)
[4] Kaipio, J.; Somersalo, E., Statistical and Computational Inverse Problems (2005), Springer: Springer New York · Zbl 1068.65022
[5] Wang, J.; Zabaras, N., A Bayesian inference approach to the inverse heat conduction problem, Int. J. Heat Mass Transfer, 47, 3927-3941 (2004) · Zbl 1070.80002
[6] Wang, J.; Zabaras, N., Hierarchical Bayesian models for inverse problems in heat conduction, Inverse Probl., 21, 183-206 (2005) · Zbl 1060.62036
[7] Emery, A. F.; Valenti, E.; Bardot, D., Using Bayesian inference for parameter estimation when the system response and experimental conditions are measured with error and some variables are considered as nuisance variables, Meas. Sci. Technol., 18, 19-29 (2007)
[8] Jin, B.; Zou, J., A Bayesian inference approach to the ill-posed Cauchy problem of steady-state heat conduction, Int. J. Numer. Methods Eng., 76, 521-544 (2008) · Zbl 1195.80039
[9] Jin, B.; Zou, J., Augmented Tikhonov regularization, Inverse Probl., 25, 025001 (2009) · Zbl 1163.65020
[10] Jordan, M. I.; Ghahramani, Z.; Jaakkola, T. S.; Saul, L. K., An introduction to variational methods for graphical models, Mach. Learn., 37, 183-233 (1999) · Zbl 0945.68164
[11] Attias, H., A variational Bayesian framework for graphical models, (Solla, S. A.; Leen, T. K.; Maller, K. R., Advances in Neural Information Processing Systems, vol. 12 (2000), MIT Press: MIT Press Cambridge), 209-215
[13] Eberly, L. E.; Casella, G., Estimating Bayesian credible intervals, J. Stat. Plan. Infer., 112, 115-132 (2003) · Zbl 1032.62023
[14] Gilks, W. R.; Richardson, S.; Spiegelhalter, D. J., Markov Chain Monte Carlo in Practice (1996), Chapman & Hall · Zbl 0832.00018
[15] Liu, J. S., Monte Carlo Strategies in Scientific Computing (2008), Springer · Zbl 1132.65003
[16] Sato, M.; Yoshioka, T.; Kajihara, S.; Toyama, K.; Goda, N.; Doya, K.; Kawato, M., Hierarchical Bayesian estimation for MEG inverse problem, NeuroImage, 23, 806-826 (2004)
[17] Babacan, S. D.; Molina, R.; Katsaggelos, A. K., Parameter estimation in TV image restoration using variational distribution approximation, IEEE Trans. Image Proc., 17, 326-339 (2008)
[18] Quinn, A.; Šmídl V, V., The Variational Bayes Method in Signal Processing (2005), Springer: Springer Berlin
[19] Gibbs, A. L.; Su, F. E., On choosing and bounding probability metrics, Int. Stat. Rev., 70, 419-435 (2002) · Zbl 1217.62014
[20] Resmerita, E.; Anderssen, R. S., Joint additive Kullback-Leibler residual minimization and regularization for linear inverse problems, Math. Methods Appl. Sci., 30, 1527-1544 (2007) · Zbl 1132.47008
[21] Eggermont, P. P.B., Maximum entropy regularization for Fredholm integral equations of the first kind, SIAM J. Math. Anal., 24, 1557-1576 (1993) · Zbl 0791.65099
[22] Hansen, P. C., Rank-Deficient and Discrete Ill-Posed Problems (1998), SIAM: SIAM Philadelphia
[23] Engl, H. W.; Hanke, M.; Neubauer, A., Regularization of Inverse Problems (1996), Kluwer: Kluwer Dordrecht · Zbl 0859.65054
[25] Daubechies, I.; Dfriese, M.; De Mol, C., An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Commun. Pure Appl. Math., 57, 1413-1457 (2004) · Zbl 1077.65055
[27] Chappell, M. A.; Groves, A. R.; Whitcher, B.; Woolrich, M. W., Variational Bayesian inference for a nonlinear forward model, IEEE Trans. Signal Proc., 57, 223-236 (2009) · Zbl 1391.94168
[28] Jin, B.; Marin, L., The plane wave method for some inverse problems associated with Helmholtz-type equations, Eng. Anal. Bound. Elem., 32, 223-240 (2008) · Zbl 1244.65163
[29] Osman, A. M.; Beck, J. V., Nonlinear inverse problem for the estimation of time-and-space dependent heat transfer coefficients, J. Thermophys., 3, 146-152 (1988)
[31] Hon, Y. C.; Wei, T., Backus-Gilbert algorithm for the Cauchy problem of the Laplace equation, Inverse Probl., 17, 261-271 (2001) · Zbl 0980.35167
[32] Jin, B., Fast Bayesian approach for parameter estimation, Int. J. Numer. Methods Eng., 76, 230-252 (2008) · Zbl 1195.65199
[33] Isakov, V., Inverse Problems for Partial Differential Equations (2006), Springer: Springer New York · Zbl 1092.35001
[35] Lee, H. K.H.; Higdon, D. M.; Bi, Z.; Ferreira, M. A.R.; West, M., Markov random field models for high-dimensional parameters in simulation of fluid flow in porous media, Technometrics, 44, 230-241 (2002)
[36] Haario, H.; Laine, M.; Mira, A.; Saksman, E., DRAM: efficient adaptive MCMC, Stat. Comput., 16, 339-354 (2006)
[37] Bazaraa, M.; Sherali, H.; Shetty, C., Nonlinear Programming: Theory and Algorithms (1993), Wiley: Wiley New York · Zbl 0774.90075
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.