×

zbMATH — the first resource for mathematics

Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. (English) Zbl 1415.68175
Summary: We introduce physics-informed neural networks – neural networks that are trained to solve supervised learning tasks while respecting any given laws of physics described by general nonlinear partial differential equations. In this work, we present our developments in the context of solving two main classes of problems: data-driven solution and data-driven discovery of partial differential equations. Depending on the nature and arrangement of the available data, we devise two distinct types of algorithms, namely continuous time and discrete time models. The first type of models forms a new family of data-efficient spatio-temporal function approximators, while the latter type allows the use of arbitrarily accurate implicit Runge-Kutta time stepping schemes with unlimited number of stages. The effectiveness of the proposed framework is demonstrated through a collection of classical problems in fluids, quantum mechanics, reaction-diffusion systems, and the propagation of nonlinear shallow-water waves.

MSC:
68T05 Learning and adaptive systems in artificial intelligence
35K57 Reaction-diffusion equations
35Q55 NLS equations (nonlinear Schrödinger equations)
35R30 Inverse problems for PDEs
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Krizhevsky, A.; Sutskever, I.; Hinton, G. E., Imagenet classification with deep convolutional neural networks, (Advances in Neural Information Processing Systems, (2012)), 1097-1105
[2] Lake, B. M.; Salakhutdinov, R.; Tenenbaum, J. B., Human-level concept learning through probabilistic program induction, Science, 350, 1332-1338, (2015) · Zbl 1355.68230
[3] Alipanahi, B.; Delong, A.; Weirauch, M. T.; Frey, B. J., Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning, Nat. Biotechnol., 33, 831-838, (2015)
[4] Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Inferring solutions of differential equations using noisy multi-fidelity data, J. Comput. Phys., 335, 736-746, (2017) · Zbl 1382.65229
[5] Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Machine learning of linear differential equations using Gaussian processes, J. Comput. Phys., 348, 683-693, (2017) · Zbl 1380.68339
[6] Owhadi, H., Bayesian numerical homogenization, Multiscale Model. Simul., 13, 812-828, (2015) · Zbl 1322.35002
[7] Rasmussen, C. E.; Williams, C. K., Gaussian Processes for Machine Learning, vol. 1, (2006), MIT Press: MIT Press Cambridge
[8] Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Numerical Gaussian processes for time-dependent and non-linear partial differential equations, (2017)
[9] Raissi, M.; Karniadakis, G. E., Hidden physics models: machine learning of nonlinear partial differential equations, (2017)
[10] Owhadi, H.; Scovel, C.; Sullivan, T., Brittleness of Bayesian inference under finite information in a continuous world, Electron. J. Stat., 9, 1-79, (2015) · Zbl 1305.62123
[11] Hornik, K.; Stinchcombe, M.; White, H., Multilayer feedforward networks are universal approximators, Neural Netw., 2, 359-366, (1989) · Zbl 1383.92015
[12] Baydin, A. G.; Pearlmutter, B. A.; Radul, A. A.; Siskind, J. M., Automatic differentiation in machine learning: a survey, (2015)
[13] Basdevant, C.; Deville, M.; Haldenwang, P.; Lacroix, J.; Ouazzani, J.; Peyret, R.; Orlandi, P.; Patera, A., Spectral and finite difference solutions of the Burgers equation, Comput. Fluids, 14, 23-41, (1986) · Zbl 0612.76031
[14] Rudy, S. H.; Brunton, S. L.; Proctor, J. L.; Kutz, J. N., Data-driven discovery of partial differential equations, Sci. Adv., 3, (2017)
[15] Lagaris, I. E.; Likas, A.; Fotiadis, D. I., Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., 9, 987-1000, (1998)
[16] Psichogios, D. C.; Ungar, L. H., A hybrid neural network-first principles approach to process modeling, AIChE J., 38, 1499-1511, (1992)
[17] Wang, J.-X.; Wu, J.; Ling, J.; Iaccarino, G.; Xiao, H., A comprehensive physics-informed machine learning framework for predictive turbulence modeling, (2017)
[18] Zhu, Y.; Zabaras, N., Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification, (2018)
[19] Hagge, T.; Stinis, P.; Yeung, E.; Tartakovsky, A. M., Solving differential equations with unknown constitutive relations as recurrent neural networks, (2017)
[20] Tripathy, R.; Bilionis, I., Deep UQ: learning deep neural network surrogate models for high dimensional uncertainty quantification, (2018)
[21] Vlachas, P. R.; Byeon, W.; Wan, Z. Y.; Sapsis, T. P.; Koumoutsakos, P., Data-driven forecasting of high-dimensional chaotic systems with long-short term memory networks, (2018) · Zbl 1402.92030
[22] Parish, E. J.; Duraisamy, K., A paradigm for data-driven predictive modeling using field inversion and machine learning, J. Comput. Phys., 305, 758-774, (2016) · Zbl 1349.76006
[23] Duraisamy, K.; Zhang, Z. J.; Singh, A. P., New approaches in turbulence and transition modeling using data-driven techniques, (53rd AIAA Aerospace Sciences Meeting, (2018)), 1284
[24] Ling, J.; Kurzawski, A.; Templeton, J., Reynolds averaged turbulence modelling using deep neural networks with embedded invariance, J. Fluid Mech., 807, 155-166, (2016) · Zbl 1383.76175
[25] Zhang, Z. J.; Duraisamy, K., Machine learning methods for data-driven turbulence modeling, (22nd AIAA Computational Fluid Dynamics Conference, (2015)), 2460
[26] Milano, M.; Koumoutsakos, P., Neural network modeling for near wall turbulent flow, J. Comput. Phys., 182, 1-26, (2002) · Zbl 1090.76535
[27] Perdikaris, P.; Venturi, D.; Karniadakis, G. E., Multifidelity information fusion algorithms for high-dimensional systems and massive data sets, SIAM J. Sci. Comput., 38, B521-B538, (2016) · Zbl 1342.62110
[28] Rico-Martinez, R.; Anderson, J.; Kevrekidis, I., Continuous-time nonlinear signal processing: a neural network based approach for gray box identification, (Neural Networks for Signal Processing IV. Proceedings of the 1994 IEEE Workshop, (1994), IEEE), 596-605
[29] Ling, J.; Templeton, J., Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty, Phys. Fluids, 27, (2015)
[30] Lin, H. W.; Tegmark, M.; Rolnick, D., Why does deep and cheap learning work so well?, J. Stat. Phys., 168, 1223-1247, (2017) · Zbl 1373.82061
[31] Kondor, R., N-body networks: a covariant hierarchical neural network architecture for learning atomic potentials, (2018)
[32] Kondor, R.; Trivedi, S., On the generalization of equivariance and convolution in neural networks to the action of compact groups, (2018)
[33] Hirn, M.; Mallat, S.; Poilvert, N., Wavelet scattering regression of quantum chemical energies, Multiscale Model. Simul., 15, 827-863, (2017) · Zbl 1365.42030
[34] Mallat, S., Understanding deep convolutional networks, Philos. Trans. R. Soc. A, 374, (2016)
[35] Liu, D. C.; Nocedal, J., On the limited memory BFGS method for large scale optimization, Math. Program., 45, 503-528, (1989) · Zbl 0696.90048
[36] Goodfellow, I.; Bengio, Y.; Courville, A., Deep Learning, (2016), MIT Press · Zbl 1373.68009
[37] Kingma, D.; Ba, J., Adam: a method for stochastic optimization, (2014)
[38] A. Choromanska, M. Henaff, M. Mathieu, G.B. Arous, Y. LeCun, The loss surfaces of multilayer networks, in: Artificial Intelligence and Statistics, pp. 192-204.
[39] Shwartz-Ziv, R.; Tishby, N., Opening the black box of deep neural networks via information, (2017)
[40] Driscoll, T. A.; Hale, N.; Trefethen, L. N., Chebfun Guide, (2014)
[41] Stein, M., Large sample properties of simulations using Latin hypercube sampling, Technometrics, 29, 143-151, (1987) · Zbl 0627.62010
[42] Snoek, J.; Larochelle, H.; Adams, R. P., Practical bayesian optimization of machine learning algorithms, (Advances in Neural Information Processing Systems, (2012)), 2951-2959
[43] Bungartz, H.-J.; Griebel, M., Sparse grids, Acta Numer., 13, 147-269, (2004) · Zbl 1118.65388
[44] Sloan, I. H.; Woźniakowski, H., When are quasi-Monte Carlo algorithms efficient for high dimensional integrals?, J. Complex., 14, 1-33, (1998) · Zbl 1032.65011
[45] Iserles, A., A First Course in the Numerical Analysis of Differential Equations, vol. 44, (2009), Cambridge University Press
[46] Von Kármán, T., Aerodynamics, vol. 9, (1963), McGraw-Hill: McGraw-Hill New York
[47] Karniadakis, G.; Sherwin, S., Spectral/hp Element Methods for Computational Fluid Dynamics, (2013), Oxford University Press · Zbl 1256.76003
[48] Dauxois, T., Fermi, Pasta, Ulam and a mysterious lady, (2008)
[49] Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G. S.; Davis, A.; Dean, J.; Devin, M., Tensorflow: large-scale machine learning on heterogeneous distributed systems, (2016)
[50] Brunton, S. L.; Proctor, J. L.; Kutz, J. N., Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci., 113, 3932-3937, (2016) · Zbl 1355.94013
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.