A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics. (English) Zbl 07340350

Summary: We present the application of a class of deep learning, known as Physics Informed Neural Networks (PINN), to inversion and surrogate modeling in solid mechanics. We explain how to incorporate the momentum balance and constitutive relations into PINN, and explore in detail the application to linear elasticity, and illustrate its extension to nonlinear problems through an example that showcases von Mises elastoplasticity. While common PINN algorithms are based on training one deep neural network (DNN), we propose a multi-network model that results in more accurate representation of the field variables. To validate the model, we test the framework on synthetic data generated from analytical and numerical reference solutions. We study convergence of the PINN model, and show that Isogeometric Analysis (IGA) results in superior accuracy and convergence characteristics compared with classic low-order Finite Element Method (FEM). We also show the applicability of the framework for transfer learning, and find vastly accelerated convergence during network re-training. Finally, we find that honoring the physics leads to improved robustness: when trained only on a few parameters, we find that the PINN model can accurately predict the solution for a wide range of parameters new to the network – thus pointing to an important application of this framework to sensitivity analysis and surrogate modeling.


74-XX Mechanics of deformable solids
92-XX Biology and other natural sciences
Full Text: DOI arXiv


[1] Bishop, C. M., Pattern Recognition and Machine Learning (2006), Springer-Verlag: Springer-Verlag Berlin, Heidelberg, URL: https://www.springer.com/gp/book/9780387310732. doi: https://dl.acm.org/doi/book/10.5555/1162264 · Zbl 1107.68072
[2] LeCun, Y.; Bengio, Y.; Hinton, G., Deep learning, Nature, 521, 7553, 436-444 (2015)
[3] Goodfellow, I.; Bengio, Y.; Courville, A., Deep Learning, 800 (2016), MIT press, URL: https://www.deeplearningbook.org. doi:https://dl.acm.org/doi/book/10.5555/3086952
[4] Yoon, C. E.; O’Reilly, O.; Bergen, K. J.; Beroza, G. C., Earthquake detection through computationally efficient similarity search, Sci. Adv., 1, 11, Article e1501057 pp. (2015), URL: http://advances.sciencemag.org/lookup/doi/10.1126/sciadv.1501057
[5] Bergen, K. J.; Johnson, P. A.; de Hoop, M. V.; Beroza, G. C., Machine learning for data-driven discovery in solid earth geoscience, Science, 363, 6433 (2019), URL: https://science.sciencemag.org/content/363/6433/eaau0323. arXiv:https://science.sciencemag.org/content/363/6433/eaau0323.full.pdf
[6] DeVries, P. M.; Viégas, F.; Wattenberg, M.; Meade, B. J., Deep learning of aftershock patterns following large earthquakes, Nature, 560, 7720, 632-634 (2018)
[7] Kong, Q.; Trugman, D. T.; Ross, Z. E.; Bianco, M. J.; Meade, B. J.; Gerstoft, P., Machine learning in seismology: turning data into insights, Seismol. Res. Lett., 90, 1, 3-14 (2018)
[8] Ren, C. X.; Dorostkar, O.; Rouet-Leduc, B.; Hulbert, C.; Strebel, D.; Guyer, R. A.; Johnson, P. A.; Carmeliet, J., Machine learning reveals the state of intermittent frictional dynamics in a sheared granular fault, Geophys. Res. Lett., 46, 13, 7395-7403 (2019), URL: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2019GL082706. arXiv:https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2019GL082706
[9] Pilania, G.; Wang, C.; Jiang, X.; Rajasekaran, S.; Ramprasad, R., Accelerating materials property predictions using machine learning, Sci. Rep., 3, 1-6 (2013)
[10] Butler, K. T.; Davies, D. W.; Cartwright, H.; Isayev, O.; Walsh, A., Machine learning for molecular and materials science, Nature, 559, 7715, 547-555 (2018)
[11] Shi, Z.; Tsymbalov, E.; Dao, M.; Suresh, S.; Shapeev, A.; Li, J., Deep elastic strain engineering of bandgap through machine learning, Proc. Natl. Acad. Sci., 116, 10, 4117-4122 (2019), URL: http://www.pnas.org/lookup/doi/10.1073/pnas.1818555116
[12] Brunton, S. L.; Kutz, J. N., Methods for data-driven multiscale model discovery for materials, J. Phys. Mater., 2, 4, Article 044002 pp. (2019)
[13] Brenner, M. P.; Eldredge, J. D.; Freund, J. B., Perspective on machine learning for advancing fluid mechanics, Phys. Rev. Fluids, 4, 10, Article 100501 pp. (2019), URL: https://link.aps.org/doi/10.1103/PhysRevFluids.4.100501
[14] Brunton, S. L.; Noack, B. R.; Koumoutsakos, P., Machine learning for fluid mechanics, Annu. Rev. Fluid Mech., 52, 1, 477-508 (2020) · Zbl 1439.76138
[15] Libbrecht, M. W.; Noble, W. S., Machine learning applications in genetics and genomics, Nature Rev. Genet., 16, 6, 321-332 (2015)
[16] Rafiei, M. H.; Adeli, H., A novel machine learning-based algorithm to detect damage in high-rise building structures, Struct. Des. Tall Special Build., 26, 18, 1-11 (2017)
[17] Sen, D.; Aghazadeh, A.; Mousavi, A.; Nagarajaiah, S.; Baraniuk, R.; Dabak, A., Data-driven semi-supervised and supervised learning algorithms for health monitoring of pipes, Mech. Syst. Signal Process., 131, 524-537 (2019)
[18] Ghaboussi, J.; Sidarta, D., New nested adaptive neural networks (NANN) for constitutive modeling, Comput. Geotech., 22, 1, 29-52 (1998)
[19] Kalidindi, S. R.; Niezgoda, S. R.; Salem, A. A., Microstructure informatics using higher-order statistics and efficient data-mining protocols, JOM, 63, 4, 34-41 (2011)
[20] Mozaffar, M.; Bostanabad, R.; Chen, W.; Ehmann, K.; Cao, J.; Bessa, M. A., Deep learning predicts path-dependent plasticity, Proc. Natl. Acad. Sci., 116, 52, 26414-26420 (2019), URL: https://www.pnas.org/content/116/52/26414. arXiv:https://www.pnas.org/content/116/52/26414.full.pdf
[21] Rudy, S.; Alla, A.; Brunton, S. L.; Kutz, J. N., Data-driven identification of parametric partial differential equations, SIAM J. Appl. Dyn. Syst., 18, 2, 643-660 (2019), URL: https://epubs.siam.org/doi/abs/10.1137/18M1191944 · Zbl 1456.65096
[22] Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707 (2019) · Zbl 1415.68175
[23] Han, J.; Jentzen, A.; E, W., Solving high-dimensional partial differential equations using deep learning, Proc. Natl. Acad. Sci., 115, 34, 8505-8510 (2018), URL: https://www.pnas.org/content/115/34/8505 · Zbl 1416.35137
[24] Bar-Sinai, Y.; Hoyer, S.; Hickey, J.; Brenner, M. P., Learning data-driven discretizations for partial differential equations, Proc. Natl. Acad. Sci., 116, 31, 15344-15349 (2019), URL: https://www.pnas.org/content/116/31/15344. arXiv:https://www.pnas.org/content/116/31/15344.full.pdf · Zbl 1431.65195
[25] Zhu, Y.; Zabaras, N.; Koutsourelakis, P.-S.; Perdikaris, P., Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data, J. Comput. Phys., 394, 56-81 (2019), URL: https://www.sciencedirect.com/science/article/pii/S0021999119303559 · Zbl 1452.68172
[26] Meade, A. J.; Fernandez, A. A., The numerical solution of linear ordinary differential equations by feed-forward neural networks, Math. Comput. Modelling, 19, 12, 1-25 (1994), URL: https://www.sciencedirect.com/science/article/pii/0895717794900957 · Zbl 0807.65079
[27] Lagaris, I. E.; Likas, A.; Fotiadis, D. I., Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., 9, 5, 987-1000 (1998), URL: https://ieeexplore.ieee.org/document/712178
[28] Lagaris, I. E.; Likas, A. C.; Papageorgiou, D. G., Neural-network methods for boundary value problems with irregular boundaries, IEEE Trans. Neural Netw., 11, 5, 1041-1049 (2000), URL: https://ieeexplore.ieee.org/document/870037
[29] J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, Y. Bengio, Theano: a CPU and GPU math expression compiler, in: Proceedings of the Python for Scientific Computing Conference (SciPy), Vol. 4, Austin, TX, 2010.
[30] Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; Kudlur, M.; Levenberg, J.; Monga, R.; Moore, S.; Murray, D. G.; Steiner, B.; Tucker, P.; Vasudevan, V.; Warden, P.; Wicke, M.; Yu, Y.; Zheng, X., TensorFlow: A system for large-scale machine learning, (12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) (2016), USENIX Association: USENIX Association Savannah, GA), 265-283, URL: https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi
[31] Lange, S.; Gabel, T.; Riedmiller, M., Reinforcement learning, (Wiering, M.; van Otterlo, M., Adaptation, Learning, and Optimization. Adaptation, Learning, and Optimization, Adaptation, Learning, and Optimization, 12 (2012), Springer Berlin Heidelberg: Springer Berlin Heidelberg Berlin, Heidelberg), 45-73, URL: http://link.springer.com/10.1007/978-3-642-27645-3
[32] Hughes, T.; Cottrell, J.; Bazilevs, Y., Isogeometric analysis: CAD, finite elements, NURBS, exact geometry and mesh refinement, Comput. Methods Appl. Mech. Engrg., 194, 39, 4135-4195 (2005), URL: http://www.sciencedirect.com/science/article/pii/S0045782504005171 · Zbl 1151.74419
[33] Cottrell, J. A.; Hughes, T. J.R.; Bazilevs, Y., Isogeometric Analysis: Toward Integration of CAD and FEA (2009), John Wiley & Sons, doi:https://dl.acm.org/doi/book/10.5555/1816404 · Zbl 1378.65009
[34] Taylor, M. E.; Stone, P., Transfer learning for reinforcement learning domains: A survey, J. Mach. Learn. Res., 10, Jul, 1633-1685 (2009), URL: http://www.jmlr.org/papers/v10/taylor09a.html. doi:https://dl.acm.org/doi/10.5555/1577069.1755839 · Zbl 1235.68196
[35] Baydin, A. G.; Pearlmutter, B. A.; Radul, A. A.; Siskind, J. M., Automatic differentiation in machine learning: a survey, J. Mach. Learn. Res., 18, 1, 5595-5637 (2017), URL: https://dl.acm.org/doi/abs/10.5555/3122009.3242010
[36] Chen, T.; Li, M.; Li, Y.; Lin, M.; Wang, N.; Wang, M.; Xiao, T.; Xu, B.; Zhang, C.; Zhang, Z., MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems (2015), arXiv:1512.01274
[37] Chollet, F., Keras (2015)
[38] Kingma, D. P.; Ba, J., Adam: A method for stochastic optimization (2014), arXiv:1412.6980
[39] Duchi, J.; Hazan, E.; Singer, Y., Adaptive subgradient methods for online learning and stochastic optimization, J. Mach. Learn. Res., 12, Jul, 2121-2159 (2011), URL: http://jmlr.org/papers/v12/duchi11a.html · Zbl 1280.68164
[40] Haghighat, E.; Juanes, R., SciANN: A keras/tensorflow wrapper for scientific computations and physics-informed deep learning using artificial neural networks, Comput. Methods Appl. Mech. Engrg., 373, 113552 (2021)
[41] COMSOL Multiphysics User’s Guide (2020), COMSOL: COMSOL Stockholm, Sweden
[42] Bazilevs, Y.; Calo, V. M.; Cottrell, J. A.; Evans, J. A.; Hughes, T. J.R.; Lipton, S.; Scott, M. A.; Sederberg, T. W., Isogeometric analysis using T-splines, Comput. Methods Appl. Mech. Engrg., 199, 5-8, 229-263 (2010), URL: https://www.sciencedirect.com/science/article/pii/S0045782509000875 · Zbl 1227.74123
[43] Simo, J. C.; Hughes, T. J.R., (Computational Inelasticity. Computational Inelasticity, Interdisciplinary Applied Mathematics, vol. 7 (1998), Springer: Springer New York) · Zbl 0934.74003
[44] Zienkiewicz, O. C.; Valliappan, S.; King, I., Elasto-plastic solutions of engineering problems ‘initial stress’, finite element approach, Internat. J. Numer. Methods Engrg., 1, 1, 75-100 (1969) · Zbl 0247.73087
[45] Smith, S. L.; Kindermans, P.-J.; Ying, C.; Le, Q. V., Don’t decay the learning rate, increase the batch size (2017), arXiv preprint arXiv:1711.00489
[46] Wang, S.; Teng, Y.; Perdikaris, P., Understanding and mitigating gradient pathologies in physics-informed neural networks (2020), arXiv preprint arXiv:2001.04536
[47] Wang, S.; Yu, X.; Perdikaris, P., When and why PINNs fail to train: A neural tangent kernel perspective (2020), arXiv preprint arXiv:2007.14527
[48] Rahaman, N.; Baratin, A.; Arpit, D.; Draxler, F.; Lin, M.; Hamprecht, F.; Bengio, Y.; Courville, A., On the spectral bias of neural networks, (Chaudhuri, K.; Salakhutdinov, R., Proceedings of the 36th International Conference on Machine Learning. Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 97 (2019), PMLR), 5301-5310, URL: http://proceedings.mlr.press/v97/rahaman19a.html
[49] Jagtap, A. D.; Karniadakis, G. E., Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, Commun. Comput. Phys., 28, 5, 2002-2041 (2020)
[50] Haghighat, E.; Bekar, A. C.; Madenci, E.; Juanes, R., A nonlocal physics-informed deep learning framework using the peridynamic differential operator (2020), arXiv preprint arXiv:2006.00446
[51] Wang, S.; Wang, H.; Perdikaris, P., On the eigenvector bias of fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks (2020), arXiv preprint arXiv:2012.10047
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.