×

SciANN: a keras/tensorflow wrapper for scientific computations and physics-informed deep learning using artificial neural networks. (English) Zbl 07337817

Summary: In this paper, we introduce SciANN, a Python package for scientific computing and physics-informed deep learning using artificial neural networks. SciANN uses the widely used deep-learning packages TensorFlow and Keras to build deep neural networks and optimization models, thus inheriting many of Keras’s functionalities, such as batch optimization and model reuse for transfer learning. SciANN is designed to abstract neural network construction for scientific computations and solution and discovery of partial differential equations (PDE) using the physics-informed neural networks (PINN) architecture, therefore providing the flexibility to set up complex functional forms. We illustrate, in a series of examples, how the framework can be used for curve fitting on discrete data, and for solution and discovery of PDEs in strong and weak forms. We summarize the features currently available in SciANN, and also outline ongoing and future developments.

MSC:

68-XX Computer science
65-XX Numerical analysis
PDF BibTeX XML Cite
Full Text: DOI arXiv

References:

[1] Bishop, C. M., Pattern Recognition and Machine Learning (2006), Springer, URL http://www.springer.com/us/book/9780387310732 · Zbl 1107.68072
[2] Krizhevsky, A.; Sutskever, I.; Hinton, G. E., ImageNet classification with deep convolutional neural networks, (Pereira, F.; Burges, C. J.C.; Bottou, L.; Weinberger, K. Q., Advances in Neural Information Processing Systems 25 (2012), MIT Press), 1097-1105, URL http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
[3] LeCun, Y.; Bengio, Y.; Hinton, G., Deep learning, Nature, 521, 7553, 436-444 (2015)
[4] Jannach, D.; Zanker, M.; Felfernig, A.; Friedrich, G., Recommender Systems: An Introduction (2010), Cambridge University Press
[5] Zhang, S.; Yao, L.; Sun, A.; Tay, Y., Deep learning based recommender system: A survey and new perspectives, ACM Comput. Surv., 52, 1, 1-38 (2019)
[6] Graves, A.; Mohamed, A.-R.; Hinton, G., Speech recognition with deep recurrent neural networks, (2013 IEEE International Conference on Acoustics, Speech and Signal Processing (2013), IEEE), 6645-6649
[7] Bojarski, M.; Del Testa, D.; Dworakowski, D.; Firner, B.; Flepp, B.; Goyal, P.; Jackel, L. D.; Monfort, M.; Muller, U.; Zhang, J., End to end learning for self-driving cars (2016), arXiv preprint arXiv:1604.07316
[8] Miotto, R.; Wang, F.; Wang, S.; Jiang, X.; Dudley, J. T., Deep learning for healthcare: review, opportunities and challenges, Brief. Bioinform., 19, 6, 1236-1246 (2018)
[9] Goodfellow, I.; Bengio, Y.; Courville, A., Deep Learning (2016), MIT Press, URL https://www.deeplearningbook.org · Zbl 1373.68009
[10] Kong, Q.; Trugman, D. T.; Ross, Z. E.; Bianco, M. J.; Meade, B. J.; Gerstoft, P., Machine learning in seismology: turning data into insights, Seismol. Res. Lett., 90, 1, 3-14 (2018)
[11] Ross, Z. E.; Trugman, D. T.; Hauksson, E.; Shearer, P. M., Searching for hidden earthquakes in Southern California, Science, 364, 6442, 767-771 (2019)
[12] Bergen, K. J.; Johnson, P. A.; de Hoop, M. V.; Beroza, G. C., Machine learning for data-driven discovery in solid earth geoscience, Science, 363, 6433, eaau0323 (2019)
[13] Brenner, M. P.; Eldredge, J. D.; Freund, J. B., Perspective on machine learning for advancing fluid mechanics, Phys. Rev. Fluids, 4, 10, Article 100501 pp. (2019)
[14] Brunton, S. L.; Noack, B. R.; Koumoutsakos, P., Machine learning for fluid mechanics, Annu. Rev. Fluid Mech., 52, 1, 477-508 (2020) · Zbl 1439.76138
[15] Dana, S.; Wheeler, M. F., A machine learning accelerated \(\operatorname{FE}^2\) homogenization algorithm for elastic solids (2020), arXiv:2003.11372
[16] Tartakovsky, A. M.; Marrero, C. O.; Perdikaris, P.; Tartakovsky, G. D.; Barajas-Solano, D., Learning parameters and constitutive relationships with physics informed deep neural networks (2018), arXiv:1808.03398
[17] Xu, K.; Huang, D. Z.; Darve, E., Learning constitutive relations using symmetric positive definite neural networks, 1-31 (2020), arXiv:2004.00265.
[18] Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707 (2019) · Zbl 1415.68175
[19] Raissi, M.; Yazdani, A.; Karniadakis, G. E., Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations, Science, 367, 6481, 1026-1030 (2020)
[20] Haghighat, E.; Raissi, M.; Moure, A.; Gomez, H.; Juanes, R., A deep learning framework for solution and discovery in solid mechanics (2020), arXiv:2003.02751.
[21] Rudy, S.; Alla, A.; Brunton, S. L.; Kutz, J. N., Data-driven identification of parametric partial differential equations, SIAM J. Appl. Dyn. Syst., 18, 2, 643-660 (2019) · Zbl 1456.65096
[22] J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, Y. Bengio, Theano: a CPU and GPU math expression compiler, in: Proceedings of the Python for Scientific Computing Conference (SciPy), Vol. 4, Austin, TX, 2010.
[23] Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; Kudlur, M.; Levenberg, J.; Monga, R.; Moore, S.; Murray, D. G.; Steiner, B.; Tucker, P.; Vasudevan, V.; Warden, P.; Wicke, M.; Yu, Y.; Zheng, X., Tensorflow: A system for large-scale machine learning, (12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) (2016), USENIX Association: USENIX Association Savannah, GA), 265-283, URL https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi
[24] Chen, T.; Li, M.; Li, Y.; Lin, M.; Wang, N.; Wang, M.; Xiao, T.; Xu, B.; Zhang, C.; Zhang, Z., MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems (2015), arXiv:1512.01274
[25] Chollet, F., Deep Learning with Python (2017), Manning Publications Company, URL https://books.google.ca/books?id=Yo3CAQAACAAJ
[26] Güne, A.; Baydin, G.; Pearlmutter, B. A.; Siskind, J. M., Automatic differentiation in machine learning: a survey, J. Mach. Learn. Res., 18, 1-43 (2018), URL http://www.jmlr.org/papers/volume18/17-468/17-468.pdf · Zbl 06982909
[27] Kharazmi, E.; Zhang, Z.; Karniadakis, G. E., Variational physics-informed neural networks for solving partial differential equations, 1-24 (2019), arXiv:1912.00873.
[28] Hornik, K.; Stinchcombe, M.; White, H., Multilayer feed-forward networks are universal approximators, Neural Netw., 2, 5, 359-366 (1989) · Zbl 1383.92015
[29] Cybenko, G., Approximation by superpositions of a sigmoidal function, Math. Control Signals Systems, 2, 4, 303-314 (1989) · Zbl 0679.94019
[30] Hornik, K., Approximation capabilities of multilayer feed-forward networks, Neural Netw., 4, 2, 251-257 (1991)
[31] Rumelhart, D. E.; Hinton, G. E.; Williams, R. J., Learning representations by back-propagating errors, Nature, 323, 6088, 533-536 (1986) · Zbl 1369.68284
[32] Dafermos, C. M., Hyperbolic Conservation Laws in Continuum Physics (2000), Springer-Verlag: Springer-Verlag Berlin · Zbl 0940.35002
[33] Raissi, M., Deep hidden physics models: Deep learning of nonlinear partial differential equations, J. Mach. Learn. Res., 19, 1-24 (2018), arXiv:1801.06637 · Zbl 1439.68021
[34] Simo, J. C.; Hughes, T. J.R., (Computational Inelasticity. Computational Inelasticity, Interdisciplinary Applied Mathematics, vol. 7 (1998), Springer: Springer New York) · Zbl 0934.74003
[35] Zienkiewicz, O.; Valliappan, S.; King, I., Elasto-plastic solutions of engineering problems ‘initial stress’, finite element approach, Internat. J. Numer. Methods Engrg., 1, 1, 75-100 (1969) · Zbl 0247.73087
[36] COMSOL Multiphysics User’s Guide (2020), COMSOL: COMSOL Stockholm, Sweden
[37] Weinan, E.; Yu, B., The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat., 6, 1, 1-14 (2018), arXiv:1710.00211 · Zbl 1392.35306
[38] Berg, J.; Nyström, K., A unified deep artificial neural network approach to partial differential equations in complex geometries, Neurocomputing, 317, 28-41 (2018), arXiv:1711.06464
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.