Neural network as a function approximator and its application in solving differential equations.

*(English)*Zbl 1416.65060Summary: A neural network (NN) is a powerful tool for approximating bounded continuous functions in machine learning. The NN provides a framework for numerically solving ordinary differential equations (ODEs) and partial differential equations (PDEs) combined with the automatic differentiation (AD) technique. In this work, we explore the use of NN for the function approximation and propose a universal solver for ODEs and PDEs. The solver is tested for initial value problems and boundary value problems of ODEs, and the results exhibit high accuracy for not only the unknown functions but also their derivatives. The same strategy can be used to construct a PDE solver based on collocation points instead of a mesh, which is tested with the Burgers equation and the heat equation (i.e., the Laplace equation).

##### MSC:

65D15 | Algorithms for approximation of functions |

##### Keywords:

neural network (NN); function approximation; ordinary differential equation (ODE) solver; partial differential equation (PDE) solver
PDF
BibTeX
XML
Cite

\textit{Z. Liu} et al., AMM, Appl. Math. Mech., Engl. Ed. 40, No. 2, 237--248 (2019; Zbl 1416.65060)

Full Text:
DOI

##### References:

[1] | Gerven, M.; Bohte, S., Artificial neural networks as models of neural information processing, Frontiers in Computational Neuroscience, 11, 00114, (2017) |

[2] | HAYKIN, S. S. Neural Networks and Learning Machines, 3rd ed., Prentice Hall, Englewood, New Jersey (2008) |

[3] | Krizhevsky, A.; Sutskever, I.; Hinton, G. E., Imagenet classification with deep convolutional neural networks, Proceedings of the 25th International Conference on Neural Information Processing Systems, 1, 1097-1105, (2012) |

[4] | Lecun, Y.; Bengio, Y.; Hinton, G., Deep learning, nature, 521, 436-444, (2015) |

[5] | Lake, B. M.; Salakhutdinov, R.; Tenenbaum, J. B., Human-level concept learning through probabilistic program induction, Science, 350, 1332-1338, (2015) · Zbl 1355.68230 |

[6] | Alipanahi, B.; Delong, A.; Weirauch, M. T.; Frey, B. J., Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning, Nature Biotechnology, 33, 831-838, (2015) |

[7] | Mcculloch, W. S.; Pitts, W., A logical calculus of the ideas immanent in nervous activity, The Bulletin of Mathematical Biophysics, 5, 115-133, (1943) · Zbl 0063.03860 |

[8] | Rosenblatt, F., The perceptron: a probabilistic model for information storage and organization in the brain, Psychological Review, 65, 386-408, (1958) |

[9] | ROSENBLATT, F. Principles of Neurodynamics: Perceptions and the Theory of Brain Mechanism, Spartan Books, Washington, D.C. (1961) |

[10] | MINSKY, M. and PAPERT, S. A. Perceptrons: An Introduction to Computational Geometry, MIT Press, Cambridge (1969) · Zbl 0197.43702 |

[11] | WERBOS, P. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, Ph.D. dissertation, Harvard University (1974) |

[12] | RALL, L. B. Automatic Differentiation: Techniques and Applications, Springer-Verlag, Berlin (1981) · Zbl 0473.68025 |

[13] | Baydin, A. G.; Pearlmutter, B. A.; Radul, A. A.; Siskind, J. M., Automatic differentiation in machine learning: a survey, The Journal of Machine Learning Research, 18, 1-43, (2018) · Zbl 06982909 |

[14] | RAISSI, M., PERDIKARIS, P., and KARNIADAKIS, G. E. Physics informed deep learning (part i): data-driven solutions of nonlinear partial differential equations. arXiv, arXiv: 1711.10561 (2017) https://arxiv.org/abs/1711.10561 |

[15] | Mall, S.; Chakraverty, S., Application of legendre neural network for solving ordinary differential equations, Applied Soft Computing, 43, 347-356, (2016) |

[16] | Mall, S.; Chakraverty, S., Chebyshev neural network based model for solving laneemden type equations, Applied Mathematics and Computation, 247, 100-114, (2014) · Zbl 1338.65206 |

[17] | BERG, J. and NYSTRĂ–M, K. A unified deep artificial neural network approach to partial differential equations in complex geometries. arXiv, arXiv: 1711.06464 (2017) https://arxiv.org/abs/ 1711.06464 |

[18] | Mai-Duy, N.; Tran-Cong, T., Numerical solution of differential equations using multiquadric radial basis function networks, Neural Networks, 14, 185-199, (2001) · Zbl 1047.76101 |

[19] | Jianyu, L.; Siwei, L.; Yingjian, Q.; Yaping, H., Numerical solution of elliptic partial differential equation using radial basis function neural networks, Neural Networks, 16, 729-734, (2003) |

[20] | Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; Kudlur, M., Tensorflow: a system for large-scale machine learning, The 12th USENIX Symposium on Operating Systems Design and Implementation, 16, 265-283, (2016) |

[21] | Hornik, K.; Stinchcombe, M.; White, H., Multilayer feedforward networks are universal approximators, Neural Networks, 2, 359-366, (1989) · Zbl 1383.92015 |

[22] | Cybenko, G., Approximation by superpositions of a sigmoidal function, Mathematics of Control, Signals and Systems, 2, 303-314, (1989) · Zbl 0679.94019 |

[23] | Jones, L. K., Constructive approximations for neural networks by sigmoidal functions, Proceedings of the IEEE, 78, 1586-1589, (1990) |

[24] | Carroll, S.; Dickinson, B., Construction of neural networks using the Radon transform, IEEE International Conference on Neural Networks, 1, 607-611, (1989) |

[25] | Liu, D. C.; Nocedal, J., On the limited memory BFGS method for large scale optimization, Mathematical Programming, 45, 503-528, (1989) · Zbl 0696.90048 |

[26] | Lee, C. B., Possible universal transitional scenario in a flat plate boundary layer: measurement and visualization, Physical Review E, 62, 3659-3670, (2000) |

[27] | Lee, C. B.; Wu, J. Z., Transition in wall-bounded flows, Applied Mechanics Reviews, 61, 030802, (2008) · Zbl 1146.76601 |

[28] | Lee, C. B., New features of CS solitons and the formation of vortices, Physics Letters A, 247, 397-402, (1998) |

[29] | Lee, C. B.; Fu, S., On the formation of the chain of ring-like vortices in a transitional boundary layer, Experiments in Fluids, 30, 354-357, (2001) |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.