×

Simple computational strategies for more effective physics-informed neural networks modeling of turbulent natural convection. (English) Zbl 07518100

Summary: The high expressivity and agility of physics-informed neural networks (PINNs) make them promising candidates for full fluid flow PDE modeling. An important question is whether this new paradigm, exempt from the traditional notion of discretization of the underlying operators very much connected to the flow scales resolution, is capable of sustaining high levels of turbulence. Another concern is whether it can be used as numerical substitutes to full DNS data retrieval and storage; DNS remaining so far the standard tool for validation and inter-comparison with experimental results.
We explore the use of PINNs surrogate modeling for turbulent natural convection flows, mainly relying on DNS temperature data from the fluid bulk and velocity data at some fluid boundaries. This technique depends on the minimization of a composite loss-function relying on labels and PDE residuals. We demonstrate the large computational requirements under which PINNs are capable of accurately recovering the flow hidden quantities. We then propose new techniques to mitigate the need for large training datasets. First, we propose a padding technique to better distribute some of the scattered coordinates at which PDE residuals are minimized, in particular in zones where no labels are available. We show how it comes to play as a regularization close to the training boundaries and results in a noticeable global accuracy improvement at iso-budget. We then propose a relaxation of the incompressibility condition involved in the loss function contribution related to the PDE residuals. This development drastically benefits the optimization search and results in a much improved convergence. The results obtained for Rayleigh-Bénard flow at \(Ra = 2\cdot 10^9\) are particularly impressive. With training data amounting for only 0.32% of the stored DNS dataset, the predictive accuracy of the surrogate over the entire half a billion DNS coordinates yields errors for all flow variables ranging between [0.3%–4%] in the relative \(L_2\) norm.

MSC:

76Fxx Turbulence
76Mxx Basic methods in fluid mechanics
68Txx Artificial intelligence
PDFBibTeX XMLCite
Full Text: DOI HAL

References:

[1] Kutz, J. N., Deep learning in fluid dynamics, J. Fluid Mech., 814, 1-4 (2017) · Zbl 1383.76380
[2] Brunton, S. L.; Noack, B. R.; Koumoutsakos, P., Machine learning for fluid mechanics, Annu. Rev. Fluid Mech., 52, 477-508 (2020) · Zbl 1439.76138
[3] El Garroussi, S.; Ricci, S.; De Lozzo, M.; Goutal, N.; Lucor, D., Tackling random fields non-linearities with unsupervised clustering of polynomial chaos expansion in latent space: application to global sensitivity analysis of river flooding, Stoch. Environ. Res. Risk Assess., 1-26 (2021)
[4] Cheng, S.; Argaud, J.-P.; Iooss, B.; Ponçot, A.; Lucor, D., A graph clustering approach to localization for adaptive covariance tuning in data assimilation based on state-observation mapping, Math. Geosci., 1-30 (2021)
[5] Goodfellow, I.; Bengio, Y.; Courville, A., Deep Learning (2016), MIT Press · Zbl 1373.68009
[6] Ling, J.; Jones, R.; Templeton, J., Machine learning strategies for systems with invariance properties, J. Comput. Phys., 318, 22-35 (2016) · Zbl 1349.76124
[7] Xiao, H.; Cinnella, P., Quantification of model uncertainty in RANS simulations: a review, Prog. Aerosp. Sci., 108, 1-31 (2019)
[8] Duraisamy, K.; Iaccarino, G.; Xiao, H., Turbulence modeling in the age of data, Annu. Rev. Fluid Mech., 51, 357-377 (2019) · Zbl 1412.76040
[9] Raissi, M.; Perdikaris, P.; Karniadakis, G., Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707 (2019) · Zbl 1415.68175
[10] Chillà, F.; Schumacher, J., New perspectives in turbulent Rayleigh-Bénard convection, Eur. Phys. J. E, 35, 58 (2012)
[11] Castillo-Castellanos, A.; Sergent, A.; Podvin, B.; Rossi, M., Cessation and reversals of large-scale structures in square Rayleigh-Bénard cells, J. Fluid Mech., 877, 922-954 (2019) · Zbl 1430.76299
[12] Kim, J.; Lee, C., Prediction of turbulent heat transfer using convolutional neural networks, J. Fluid Mech., 882 (2020) · Zbl 1430.76246
[13] Fonda, E.; Pandey, A.; Schumacher, J.; Sreenivasan, K. R., Deep learning in turbulent convection networks, Proc. Natl. Acad. Sci., 116, 18, 8667-8672 (2019) · Zbl 1431.76063
[14] Pandey, S.; Schumacher, J., Reservoir computing model of two-dimensional turbulent convection (2020), arXiv preprint
[15] Raissi, M.; Yazdani, A.; Karniadakis, G. E., Hidden fluid mechanics: learning velocity and pressure fields from flow visualizations, Science, 367, 6481, 1026-1030 (2020) · Zbl 1478.76057
[16] Jin, X.; Cai, S.; Li, H.; Karniadakis, G. E., NSFnets (Navier-Stokes flow nets): physics-informed neural networks for the incompressible Navier-Stokes equations (2020), arXiv preprint
[17] Meade, A.; Fernandez, A., The numerical solution of linear ordinary differential equations by feedforward neural networks, Math. Comput. Model., 19, 12, 1-25 (1994) · Zbl 0807.65079
[18] Lagaris, I. E.; Likas, A. C.; Papageorgiou, D. G., Neural-network methods for boundary value problems with irregular boundaries, IEEE Trans. Neural Netw., 11, 5, 1041-1049 (2000)
[19] Lagaris, I. E.; Likas, A.; Fotiadis, D. I., Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., 9, 5, 987-1000 (1998)
[20] McFall, K. S.; Mahan, J. R., Artificial neural network method for solution of boundary value problems with exact satisfaction of arbitrary boundary conditions, IEEE Trans. Neural Netw., 20, 8, 1221-1233 (2009)
[21] Kumar, M.; Yadav, N., Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: a survey, Comput. Math. Appl., 62, 10, 3796-3811 (2011) · Zbl 1236.65107
[22] Mall, S.; Chakraverty, S., Application of Legendre neural network for solving ordinary differential equations, Appl. Soft Comput., 43, 347-356 (2016)
[23] Berg, J.; Nyström, K., A unified deep artificial neural network approach to partial differential equations in complex geometries, Neurocomputing, 317, 28-41 (2018)
[24] Yang, C.; Yang, X.; Xiao, X., Data-driven projection method in fluid simulation, Comput. Animat. Virtual Worlds, 27, 3-4, 415-424 (2016)
[25] Haghighat, E.; Juanes, R., SciANN: a Keras/TensorFlow wrapper for scientific computations and physics-informed deep learning using artificial neural networks, Comput. Methods Appl. Mech. Eng., 373, Article 113552 pp. (2021) · Zbl 1506.65251
[26] Krishnapriyan, A.; Gholami, A.; Zhe, S.; Kirby, R.; Mahoney, M. W., Characterizing possible failure modes in physics-informed neural networks, Adv. Neural Inf. Process. Syst., 34 (2021)
[27] Wang, S.; Teng, Y.; Perdikaris, P., Understanding and mitigating gradient flow pathologies in physics-informed neural networks, SIAM J. Sci. Comput., 43, 5, A3055-A3081 (2021) · Zbl 07395804
[28] Dwivedi, V.; Parashar, N.; Srinivasan, B., Distributed physics informed neural network for data-efficient solution to partial differential equations (2019), arXiv preprint
[29] Sun, L.; Gao, H.; Pan, S.; Wang, J.-X., Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data, Comput. Methods Appl. Mech. Eng., 361, Article 112732 pp. (2020) · Zbl 1442.76096
[30] Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G. S.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Goodfellow, I.; Harp, A.; Irving, G.; Isard, M.; Jia, Y.; Jozefowicz, R.; Kaiser, L.; Kudlur, M.; Levenberg, J.; Mané, D.; Monga, R.; Moore, S.; Murray, D.; Olah, C.; Schuster, M.; Shlens, J.; Steiner, B.; Sutskever, I.; Talwar, K.; Tucker, P.; Vanhoucke, V.; Vasudevan, V.; Viégas, F.; Vinyals, O.; Warden, P.; Wattenberg, M.; Wicke, M.; Yu, Y.; Zheng, X., TensorFlow: large-scale machine learning on heterogeneous systems (2015), software available from tensorflow.org
[31] Shin, Y.; Darbon, J.; Karniadakis, G. E., On the convergence and generalization of Physics Informed Neural Networks, arXiv e-prints (2020)
[32] Mishra, S.; Molinaro, R., Estimates on the generalization error of physics informed neural networks (PINNs) for approximating PDEs (2020), arXiv preprint
[33] Rumelhart, D. E.; Hinton, G. E.; Williams, R. J., Learning representations by back-propagating errors, Nature, 323, 6088, 533-536 (1986) · Zbl 1369.68284
[34] Bengio, Y., Practical recommendations for gradient-based training of deep architectures, (Neural Networks: Tricks of the Trade (2012), Springer), 437-478
[35] Kingma, D. P.; Ba Adam, J., A method for stochastic optimization (2014), arXiv preprint
[36] Glorot, X.; Bengio, Y., Understanding the difficulty of training deep feedforward neural networks, (Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (2010)), 249-256
[37] Guermond, J.; Minev, P.; Shen, J., An overview of projection methods for incompressible flows, Comput. Methods Appl. Mech. Eng., 195, 44, 6011-6045 (2006) · Zbl 1122.76072
[38] Belkadi, M.; Sergent, A.; Fraigneau, Y.; Podvin, B., On the role of roughness valleys in turbulent Rayleigh-Bénard convection, J. Fluid Mech., 923, A6 (2021) · Zbl 1493.76056
[39] Wang, Y.; He, X.; Tong, P., Turbulent temperature fluctuations in a closed Rayleigh-Bénard convection cell, J. Fluid Mech., 874, 263-284 (2019) · Zbl 1419.76597
[40] Agrawal, A.; Lucor, D.; Fraigneau, Y.; Podvin, B.; Sergent, A., PDE-constrained neural network for turbulent Rayleigh-Bénard convection, (Workshop on Frontiers of Uncertainty Quantification in Fluid Dynamics (FrontUQ19) (September 2019)), 11-13
[41] Engels, T.; Kolomenskiy, D.; Schneider, K.; Sesterhenn, J., Numerical simulation of fluid-structure interaction with the volume penalization method, J. Comput. Phys., 281, 96-115 (2015) · Zbl 1351.76202
[42] (Bijl, H.; Lucor, D.; Mishra, S.; Schwab, C., Uncertainty Quantification in Computational Fluid Dynamics. Uncertainty Quantification in Computational Fluid Dynamics, Lecture Notes in Computational Science and Engineering, vol. 92 (2013), Springer: Springer Cham) · Zbl 1276.76005
[43] Mao, Z.; Jagtap, A. D.; Karniadakis, G. E., Physics-informed neural networks for high-speed flows, Comput. Methods Appl. Mech. Eng., 360, Article 112789 pp. (2020) · Zbl 1442.76092
[44] Mishra, S.; Rusch, T. K., Enhancing accuracy of deep learning algorithms by training with low-discrepancy sequences (2020), arXiv preprint
[45] Canevet, O.; Jose, C.; Fleuret, F., Importance sampling tree for large-scale empirical expectation, (Balcan, M. F.; Weinberger, K. Q., Proceedings of the 33rd International Conference on Machine Learning. Proceedings of the 33rd International Conference on Machine Learning, Proceedings of Machine, vol. 48 (2016), Learning Research, PMLR: Learning Research, PMLR New York, New York, USA), 1454-1462
[46] Yang, L.; Meng, X.; Karniadakis, G. E., B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data, J. Comput. Phys., 425, Article 109913 pp. (2021) · Zbl 07508507
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.