×

ConvPDE-UQ: convolutional neural networks with quantified uncertainty for heterogeneous elliptic partial differential equations on varied domains. (English) Zbl 1457.65245

The aim is to construct new solvers for partial differential equations (PDE) based on convolutional neural networks. One considers the classical Poisson equation with Dirichlet boundary condition, \(\Delta u = f\) in \(\Omega\), \( u=0\) on \(\partial\Omega\), \(\Omega\) being the two-dimensional unit disk.
The solution can be expressed by means of the Green function \(G(x,y)\), \(u(x)=\int_{\Omega}G(x,y)f(y)dy\), for \(x\in\Omega\).
A discretization procedure is applied and the purpose is to define a neural network which approximates the discretized solution mapping. The third section of the article is devoted to the presentation of the network architecture, the probabilistic training procedure, network loss functions, data generation. Numerical results, comparison of training procedures, uncertainty quantification and inference speed are shown in the fourth section and a brief summary on the developed ConvPDE-UQ framework in the fifth section. Appendix A presents results from the extension of the method to variable coefficients differential operators or Neumann boundary conditions. Appendix B deals with alternative distributions for modeling network uncertainty.

MSC:

65N75 Probabilistic methods, particle methods, etc. for boundary value problems involving PDEs
65N80 Fundamental solutions, Green’s function methods, etc. for boundary value problems involving PDEs
35J05 Laplace operator, Helmholtz equation (reduced wave equation), Poisson equation
68T05 Learning and adaptive systems in artificial intelligence
68T07 Artificial neural networks and deep learning
82C32 Neural nets applied to problems in time-dependent statistical mechanics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Lee, H.; Kang, I. S., Neural algorithm for solving differential equations, J. Comput. Phys., 91, 1, 110-131 (1990) · Zbl 0717.65062
[2] Dissanayake, M.; Phan-Thien, N., Neural-network-based approximations for solving partial differential equations, Commun. Numer. Methods Eng., 10, 3, 195-201 (1994) · Zbl 0802.65102
[3] Takeuchi, J.; Kosugi, Y., Neural network representation of finite element method, Neural Netw., 7, 2, 389-395 (1994)
[4] Lagaris, I. E.; Likas, A.; Fotiadis, D. I., Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., 9, 5, 987-1000 (1998)
[5] Lagaris, I. E.; Likas, A. C.; Papageorgiou, D. G., Neural-network methods for boundary value problems with irregular boundaries, IEEE Trans. Neural Netw., 11, 5, 1041-1049 (2000)
[6] Malek, A.; Beidokhti, R. S., Numerical solution for high order differential equations using a hybrid neural network-optimization method, Appl. Math. Comput., 183, 1, 260-271 (2006) · Zbl 1105.65340
[7] Weinan, E.; Han, J.; Jentzen, A., Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations, Commun. Math. Stat., 5, 4, 349-380 (2017) · Zbl 1382.65016
[8] Sirignano, J.; Spiliopoulos, K., Dgm: a deep learning algorithm for solving partial differential equations, J. Comput. Phys., 375, 1339-1364 (2018) · Zbl 1416.65394
[9] Tripathy, R.; Bilionis, I., Deep uq: learning deep neural network surrogate models for high dimensional uncertainty quantification, J. Comput. Phys., 375, 565-588 (2018) · Zbl 1419.68084
[10] Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics informed deep learning (part i): data-driven solutions of nonlinear partial differential equations, arXiv preprint · Zbl 1415.68175
[11] Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics informed deep learning (part ii): data-driven discovery of nonlinear partial differential equations, arXiv preprint · Zbl 1415.68175
[12] Hornik, K.; Stinchcombe, M.; White, H., Multilayer feedforward networks are universal approximators, Neural Netw., 2, 5, 359-366 (1989) · Zbl 1383.92015
[13] Hornik, K.; Stinchcombe, M.; White, H., Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks, Neural Netw., 3, 5, 551-560 (1990)
[14] Cybenko, G., Approximation by superpositions of a sigmoidal function, Math. Control Signals Syst., 2, 4, 303-314 (1989) · Zbl 0679.94019
[15] Hennig, P.; Hauberg, S., Probabilistic solutions to differential equations and their application to Riemannian statistics, (Artificial Intelligence and Statistics (2014)), 347-355
[16] Chkrebtii, O. A.; Campbell, D. A.; Calderhead, B.; Girolami, M. A., Bayesian solution uncertainty quantification for differential equations, Bayesian Anal., 11, 4, 1239-1267 (2016) · Zbl 1357.62108
[17] Wang, Y. P.; Cheng, Y.; Zhang, Z. Y.; Lin, G., Calibration of reduced-order model for a coupled Burgers equations based on pc-enkf, Math. Model. Nat. Phenom., 13, 21 (2018) · Zbl 1405.65130
[18] Wang, Y.; Hu, K.; Ren, L.; Lin, G., Optimal observations-based retrieval of topography in 2d shallow water equations using pc-enkf, J. Comput. Phys., 382, 43-60 (2019) · Zbl 1451.62165
[19] Wang, Y. P.; Ren, L. L.; Zhang, Z. Y.; Lin, G.; Xu, C., Sparsity-promoting elastic net method with rotation for high-dimensional nonlinear inverse problem, Comput. Methods Appl. Mech. Eng., 345, 263-282 (2019) · Zbl 1440.65065
[20] Gal, Y.; Ghahramani, Z., Dropout as a Bayesian approximation: representing model uncertainty in deep learning, (International Conference on Machine Learning (2016)), 1050-1059
[21] Lakshminarayanan, B.; Pritzel, A.; Blundell, C., Simple and scalable predictive uncertainty estimation using deep ensembles, (Advances in Neural Information Processing Systems (2017)), 6402-6413
[22] Kendall, A.; Gal, Y., What uncertainties do we need in Bayesian deep learning for computer vision?, (Advances in Neural Information Processing Systems (2017)), 5574-5584
[23] Zhu, Y.; Zabaras, N., Bayesian deep convolutional encoder – decoder networks for surrogate modeling and uncertainty quantification, J. Comput. Phys., 366, 415-447 (2018) · Zbl 1407.62091
[24] Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z., Rethinking the inception architecture for computer vision, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)), 2818-2826
[25] Ronneberger, O.; Fischer, P.; Brox, T., U-net: convolutional networks for biomedical image segmentation, (International Conference on Medical Image Computing and Computer-Assisted Intervention (2015), Springer), 234-241
[26] Kirby, R. C., Algorithm 839: Fiat, a new paradigm for computing finite element basis functions, ACM Trans. Math. Softw., 30, 4, 502-516 (2004) · Zbl 1070.65571
[27] Kirby, R. C.; Knepley, M. G.; Logg, A.; Scott, L. R., Optimizing the evaluation of finite element matrices, SIAM J. Sci. Comput., 27, 3, 741-758 (2005) · Zbl 1091.65114
[28] Kirby, R. C.; Logg, A., A compiler for variational forms, ACM Trans. Math. Softw., 32, 3, 417-444 (2016)
[29] Kirby, R. C.; Logg, A.; Scott, L. R.; Terrel, A. R., Topological optimization of the evaluation of finite element matrices, SIAM J. Sci. Comput., 28, 1, 224-240 (2006) · Zbl 1104.65324
[30] Kirby, R. C.; Scott, L. R., Geometric optimization of the evaluation of finite element matrices, SIAM J. Sci. Comput., 29, 2, 827-841 (2007) · Zbl 1137.65059
[31] Kirby, R. C.; Logg, A., Efficient compilation of a class of variational forms, ACM Trans. Math. Softw., 33, 3, Article 17 pp. (2007) · Zbl 1365.65257
[32] Logg, A., Automating the finite element method, Arch. Comput. Methods Eng., 14, 2, 93-138 (2007) · Zbl 1158.74048
[33] Ølgaard, K. B.; Logg, A.; Wells, G. N., Automated code generation for discontinuous Galerkin methods, SIAM J. Sci. Comput., 31, 2, 849-864 (2008) · Zbl 1189.65283
[34] Kirby, R. C.; Logg, A., Benchmarking domain-specific compiler optimizations for variational forms, ACM Trans. Math. Softw., 35, 2, 1-18 (2008)
[35] Logg, A., Efficient representation of computational meshes, Int. J. Comput. Sci. Eng., 4, 4, 283-295 (2009)
[36] Rognes, M. E.; Kirby, R. C.; Logg, A., Efficient assembly of h(div) and h(curl) conforming finite elements, SIAM J. Sci. Comput., 31, 6, 4130-4151 (2009) · Zbl 1206.65248
[37] Alnæs, M. S.; Logg, A.; Mardal, K.-A.; Skavhaug, O.; Langtangen, H. P., Unified framework for finite element assembly, Int. J. Comput. Sci. Eng., 4, 4, 231-244 (2009)
[38] Alnæs, M. S.; Mardal, K.-A., On the efficiency of symbolic computations combined with code generation for finite element methods, ACM Trans. Math. Softw., 37, 1, 6 (2010) · Zbl 1364.68375
[39] Logg, A.; Wells, G. N., Dolfin: automated finite element computing, ACM Trans. Math. Softw., 37, 2, 20 (2010) · Zbl 1364.65254
[40] Ølgaard, K. B.; Wells, G. N., Optimisations for quadrature representations of finite element tensors through automated code generation, ACM Trans. Math. Softw., 37, 1, 8 (2010) · Zbl 1364.65063
[41] Hoffman, J.; Jansson, J.; de Abreu, R. V.; Degirmenci, C.; Jansson, N.; Müller, K.; Nazarov, M.; Spühler, J. H., Unicorn: parallel adaptive finite element simulation of turbulent flow and fluid-structure interaction for deforming domains and complex geometry, Comput. Fluids, 80, 310-319 (2013) · Zbl 1284.76223
[42] Jansson, N.; Jansson, J.; Hoffman, J., Framework for massively parallel adaptive finite element computational fluid dynamics on tetrahedral meshes, SIAM J. Sci. Comput., 34, 1, C24-C41 (2012) · Zbl 1237.68244
[43] Rognes, M. E.; Ham, D. A.; Cotter, C. J.; McRae, A. T.T., Automating the solution of pdes on the sphere and other manifolds in fenics 1.2, Geosci. Model Dev., 6, 2099-2119 (2013)
[44] Alnæs, M. S.; Logg, A.; Ølgaard, K. B.; Rognes, M. E.; Wells, G. N., Unified form language: a domain-specific language for weak formulations of partial differential equations, ACM Trans. Math. Softw., 40, 2, 9 (2012) · Zbl 1308.65175
[45] Alnæs, M. S.; Blechta, J.; Hake, J.; Johansson, A.; Kehlet, B.; Logg, A.; Richardson, C.; Ring, J.; Rognes, M. E.; Wells, G. N., The fenics project version 1.5, Arch. Numer. Softw., 3, 100 (2015)
[46] Logg, A.; Mardal, K.-A.; Wells, G. N., Automated Solution of Differential Equations by the Finite Element Method (2012), Springer · Zbl 1247.65105
[47] Logg, A.; Wells, G. N.; Hake, J., DOLFIN: A C++/Python Finite Element Library (2012), Springer, Ch. 10
[48] Logg, A.; Ølgaard, K. B.; Rognes, M. E.; Wells, G. N., FFC: The FEniCS Form Compiler (2012), Springer, Ch. 11
[49] Kirby, R. C., FIAT: Numerical Construction of Finite Element Basis Functions (2012), Springer, Ch. 13
[50] Alnæs, M. S.; Mardal, K.-A., SyFi and SFC: Symbolic Finite Elements and Form Compilation (2012), Springer, Ch. 15
[51] Alnæs, M. S.; Logg, A.; Mardal, K.-A., UFC: A Finite Element Code Generation Interface (2012), Springer, Ch. 16
[52] Alnæs, M. S., UFL: A Finite Element Form Language (2012), Springer, Ch. 17
[53] Hoffman, J.; Jansson, J.; Jansson, N.; Johnson, C.; de Abreu, R. V., Turbulent Flow and Fluid-Structure Interaction (2012), Springer, Ch. 28
[54] Gilbarg, D.; Trudinger, N. S., Elliptic Partial Differential Equations of Second Order (2015), Springer · Zbl 0691.35001
[55] Kingma, D. P.; Welling, M., Auto-encoding variational Bayes, arXiv preprint
[56] Kingma, D. P.; Ba, J., Adam: a method for stochastic optimization, arXiv preprint
[57] Kundu, D., Discriminating between normal and Laplace distributions, (Advances in Ranking and Selection, Multiple Comparisons, and Reliability (2005), Springer), 65-79
[58] Copas, J., On the unimodality of the likelihood for the Cauchy distribution, Biometrika, 62, 3, 701-704 (1975) · Zbl 0321.62037
[59] Ng, E. W.; Geller, M., A table of integrals of the error functions, J. Res. Natl. Bur. Stand. B, 73, 1, 1-20 (1969) · Zbl 0184.39201
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.