×

DGM: a deep learning algorithm for solving partial differential equations. (English) Zbl 1416.65394

Summary: High-dimensional PDEs have been a longstanding computational challenge. We propose to solve high-dimensional PDEs by approximating the solution with a deep neural network which is trained to satisfy the differential operator, initial condition, and boundary conditions. Our algorithm is meshfree, which is key since meshes become infeasible in higher dimensions. Instead of forming a mesh, the neural network is trained on batches of randomly sampled time and space points. The algorithm is tested on a class of high-dimensional free boundary PDEs, which we are able to accurately solve in up to 200 dimensions. The algorithm is also tested on a high-dimensional Hamilton-Jacobi-Bellman PDE and Burgers’ equation. The deep learning algorithm approximates the general solution to the Burgers’ equation for a continuum of different boundary conditions and physical conditions (which can be viewed as a high-dimensional space). We call the algorithm a “Deep Galerkin method (DGM)” since it is similar in spirit to Galerkin methods, with the solution approximated by a neural network instead of a linear combination of basis functions. In addition, we prove a theorem regarding the approximation power of neural networks for a class of quasilinear parabolic PDEs.

MSC:

65M75 Probabilistic methods, particle methods, etc. for initial value and initial-boundary value problems involving PDEs
68T05 Learning and adaptive systems in artificial intelligence
65C05 Monte Carlo methods
65M12 Stability and convergence of numerical methods for initial value and initial-boundary value problems involving PDEs
35K59 Quasilinear parabolic equations

Software:

DGM; Adam
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Asmussen, S.; Glynn, P., Stochastic Simulation: Algorithms and Analysis (2007), Springer · Zbl 1126.65001
[2] Beck, C.; E, W.; Jentzen, A., Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations (2017)
[3] Bertsekas, D.; Tsitsiklis, J., Gradient convergence in gradient methods via errors, SIAM J. Optim., 10, 3, 627-642 (2000) · Zbl 1049.90130
[4] Boccardo, L.; Dall‘Aglio, A.; Gallouët, T.; Orsina, L., Nonlinear parabolic equations with measure data, J. Funct. Anal., 147, 237-258 (1997) · Zbl 0887.35082
[5] Boccardo, L.; Porzio, M. M.; Primo, A., Summability and existence results for nonlinear parabolic equations, Nonlinear Anal., Theory Methods Appl., 71, 304, 1-15 (2009)
[6] Bungartz, H.; Heinecke, A.; Pfluger, D.; Schraufstetter, S., Option pricing with a direct adaptive sparse grid approach, J. Comput. Appl. Math., 236, 15, 3741-3750 (2012) · Zbl 1242.91184
[7] Bungartz, H.; Griebel, M., Sparse grids, Acta Numer., 13, 174-269 (2004) · Zbl 1118.65388
[8] Chandra, P.; Singh, Y., Feedforward sigmoidal networks - equicontinuity and fault-tolerance properties, IEEE Trans. Neural Netw., 15, 6, 1350-1366 (2004)
[9] Chaudhari, P.; Oberman, A.; Osher, S.; Soatto, S.; Carlier, G., Deep Relaxation: Partial Differential Equations for Optimizing Deep Neural Networks (2017)
[10] Cerrai, S., Stationary Hamilton-Jacobi equations in Hilbert spaces and applications to a stochastic optimal control problem, SIAM J. Control Optim., 40, 3, 824-852 (2001) · Zbl 0992.60066
[11] Cybenko, G., Approximation by superposition of a sigmoidal function, Math. Control Signals Syst., 2, 303-314 (1989) · Zbl 0679.94019
[12] Davie, A.; Gaines, J., Convergence of numerical schemes for the solution of parabolic stochastic partial differential equations, Math. Comput., 70, 233, 121-134 (2000) · Zbl 0956.60064
[13] Dean, J.; Corrado, G.; Monga, R.; Chen, K.; Devin, M.; Mao, M.; Senior, A.; Tucker, P.; Yang, K.; Le, Q.; Ng, A., Large scale distributed deep networks, (Advances in Neural Information Processing Systems (2012)), 1223-1231
[14] Debussche, A.; Fuhrman, M.; Tessitore, G., Optimal control of a stochastic heat equation with boundary-noise and boundary-control, ESAIM Control Optim. Calc. Var., 13, 1, 178-205 (2007) · Zbl 1123.60052
[15] E, W.; Han, J.; Jentzen, A., Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations, Communications in Mathematics and Statistics (2017), Springer · Zbl 1382.65016
[16] Fujii, M.; Takahashi, A.; Takahashi, M., Asymptotic expansion as prior knowledge in deep learning method for high dimensional BSDEs (2017)
[17] Garcia, A. M.; Rodemich, E.; Rumsey, H.; Rosenblatt, M., A real variable lemma and the continuity of paths of some Gaussian processes, Indiana Univ. Math. J., 20, 6, 565-578 (December 1970)
[18] Glorot, X.; Bengio, Y., Understanding the difficulty of training deep feedforward neural networks, (Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (2010)), 249-256
[19] Gaines, J., Numerical Experiments with SPDEs, London Mathematical Society Lecture Note Series, 55-71 (1995) · Zbl 0829.60048
[20] Gilbarg, D.; Trudinger, N. S., Elliptic Partial Differential Equations of Second Order (1983), Springer-Verlag: Springer-Verlag Berlin, Heidelberg · Zbl 0562.35001
[21] Gyöngy, I., Lattice approximations for stochastic quasi-linear parabolic partial differential equations driven by space-time white noise I, Potential Anal., 9, 1, 1-25 (1998) · Zbl 0915.60069
[22] Heinecke, A.; Schraufstetter, S.; Bungartz, H., A highly parallel Black-Scholes solver based on adaptive sparse grids, Int. J. Comput. Math., 89, 9, 1212-1238 (2012) · Zbl 1255.91448
[23] Haugh, M.; Kogan, L., Pricing American options: a duality approach, Oper. Res., 52, 2, 258-270 (2004) · Zbl 1165.91401
[24] Hochreiter, S.; Schmidhuber, J., Long short-term memory, Neural Comput., 9, 8, 1735-1780 (1997)
[25] Hornik, K.; Stinchcombe, M.; White, H., Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks, Neural Netw., 3, 5, 551-560 (1990)
[26] Hornik, K., Approximation capabilities of multilayer feedforward networks, Neural Netw., 4, 251-257 (1991)
[27] Kingma, D.; Ba, J., ADAM: a method for stochastic optimization (2014)
[28] Ladyzenskaja, O. A.; Solonnikov, V. A.; Ural’ceva, N. N., Linear and Quasi-Linear Equations of Parabolic Type, Translations of Mathematical Monographs Reprint, vol. 23 (1988), American Mathematical Society
[29] Lagaris, I.; Likas, A.; Fotiadis, D., Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., 9, 5, 987-1000 (1998)
[30] Lagaris, I.; Likas, A.; Papageorgiou, D., Neural-network methods for boundary value problems with irregular boundaries, IEEE Trans. Neural Netw., 11, 5, 1041-1049 (2000)
[31] Lee, H., Neural algorithm for solving differential equations, J. Comput. Phys., 91, 110-131 (1990) · Zbl 0717.65062
[32] Ling, J.; Kurzawski, A.; Templeton, J., Reynolds averaged turbulence modelling using deep neural networks with embedded invariance, J. Fluid Mech., 807, 155-166 (2016) · Zbl 1383.76175
[33] Longstaff, F.; Schwartz, E., Valuing American options by simulation: a simple least-squares approach, Rev. Financ. Stud., 14, 113-147 (2001)
[34] Magliocca, M., Existence results for a Cauchy-Dirichlet parabolic problem with a repulsive gradient term, Nonlinear Anal., 166, 102-143 (2018) · Zbl 1386.35185
[35] Malek, A.; Beidokhti, R., Numerical solution for high order differential equations using a hybrid neural network-optimization method, Appl. Math. Comput., 183, 1, 260-271 (2006) · Zbl 1105.65340
[36] Masiero, F., HJB equations in infinite dimensions, J. Evol. Equ., 16, 4, 789-824 (2016) · Zbl 1361.93018
[37] Di Nardo, R.; Feo, F.; Guibé, O., Existence result for nonlinear parabolic equations with lower order terms, Anal. Appl. (Singap.), 09, 02, 161-186 (2011) · Zbl 1225.35134
[38] Petersen, P.; Voigtlaender, F., Optimal approximation of piecewise smooth functions using deep ReLU neural networks (2017)
[39] Pinkus, A., Approximation theory of the MLP model in neural networks, Acta Numer., 8, 143-195 (1999) · Zbl 0959.68109
[40] Porzio, M. M., Existence of solutions for some “noncoercive” parabolic equations, Discrete Contin. Dyn. Syst., 5, 3, 553-568 (1999) · Zbl 0953.35073
[41] Raissi, M.; Perdikaris, P.; Karniadakis, G., Physics informed deep learning (part I): data-driven solutions of nonlinear partial differential equations (2017)
[42] Raissi, M.; Perdikaris, P.; Karniadakis, G., Physics informed deep learning (part II): data-driven discovery of nonlinear partial differential equations (2017)
[43] Reisinger, C.; Wittum, G., Efficient hierarchical approximation of high-dimensional option pricing problems, SIAM J. Sci. Comput., 29, 1, 440-458 (2007) · Zbl 1151.91536
[44] Reisinger, C., Analysis of linear difference schemes in the sparse grid combination technique, IMA J. Numer. Anal., 33, 2, 544-581 (2012) · Zbl 1268.65144
[45] Rogers, L. C.G., Monte-Carlo valuation of American options, Math. Finance, 12, 3, 271-286 (2002) · Zbl 1029.91036
[46] Rudd, K., Solving Partial Differential Equations Using Artificial Neural Networks (2013), Duke University, PhD Thesis
[47] Srivastava, R.; Greff, K.; Schmidhuber, J., Training very deep networks, (Advances in Neural Information Processing Systems (2015)), 2377-2385
[48] Simon, J., Compact sets in the space \(L^p(0, T; B)\), Ann. Mat. Pura Appl., 146, 65-96 (1987) · Zbl 0629.46031
[49] Tompson, J.; Schlachter, K.; Sprechmann, P.; Perlin, K., Accelerating Eulerian fluid simulation with convolutional networks, (Proceedings of Machine Learning Research, vol. 70 (2017)), 3424-3433
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.