zbMATH — the first resource for mathematics

Recurrent neural network closure of parametric POD-Galerkin reduced-order models based on the Mori-Zwanzig formalism. (English) Zbl 1436.65093
Summary: Closure modeling based on the Mori-Zwanzig formalism has proven effective to improve the stability and accuracy of projection-based model order reduction. However, closure models are often expensive and infeasible for complex nonlinear systems. Towards efficient model reduction of general problems, this paper presents a recurrent neural network (RNN) closure of parametric POD-Galerkin reduced-order model. Based on the short time history of the reduced-order solutions, the RNN predicts the memory integral which represents the impact of the unresolved scales on the resolved scales. A conditioned long short term memory (LSTM) network is utilized as the regression model of the memory integral, in which the POD coefficients at a number of time steps are fed into the LSTM units, and the physical/geometrical parameters are fed into the initial hidden state of the LSTM. The reduced-order model is integrated in time using an implicit-explicit (IMEX) Runge-Kutta scheme, in which the memory term is integrated explicitly and the remaining right-hand-side term is integrated implicitly to improve the computational efficiency. Numerical results demonstrate that the RNN closure can significantly improve the accuracy and efficiency of the POD-Galerkin reduced-order model of nonlinear problems. The POD-Galerkin reduced-order model with the RNN closure is also shown to be capable of making accurate predictions, well beyond the time interval of the training data.
65L60 Finite element, Rayleigh-Ritz, Galerkin and collocation methods for ordinary differential equations
65F55 Numerical methods for low-rank matrix approximation; matrix compression
65L06 Multistep, Runge-Kutta and extrapolation methods for ordinary differential equations
Full Text: DOI
[1] Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G. S.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Goodfellow, I.; Harp, A.; Irving, G.; Isard, M.; Jia, Y.; Jozefowicz, R.; Kaiser, L.; Kudlur, M.; Levenberg, J.; Mané, D.; Monga, R.; Moore, S.; Murray, D.; Olah, C.; Schuster, M.; Shlens, J.; Steiner, B.; Sutskever, I.; Talwar, K.; Tucker, P.; Vanhoucke, V.; Vasudevan, V.; Viégas, F.; Vinyals, O.; Warden, P.; Wattenberg, M.; Wicke, M.; Yu, Y.; Zheng, X.; Research, G., TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (2015), Technical report
[2] Abgrall, R.; Crisovan, R., Model reduction using L 1-norm minimization as an application to nonlinear hyperbolic problems, Int. J. Numer. Methods Fluids, 87, 12, 628-651 (2018)
[3] Astrid, P.; Weiland, S.; Willcox, K.; Backx, T., Missing point estimation in models described by proper orthogonal decomposition, IEEE Trans. Autom. Control (2008) · Zbl 1367.93110
[4] Ballarin, F.; Manzoni, A.; Quarteroni, A.; Rozza, G., Supremizer stabilization of POD-Galerkin approximation of parametrized steady incompressible Navier-Stokes equations, Int. J. Numer. Methods Eng., 102, 5, 1136-1161 (2015) · Zbl 1352.76039
[5] Barrault, M.; Maday, Y.; Nguyen, N. C.; Patera, A. T., An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations, C. R. Math. (2004) · Zbl 1061.65118
[6] Benner, P.; Gugercin, S.; Willcox, K., A survey of projection-based model reduction methods for parametric dynamical systems, SIAM Rev., 57, 4, 483-531 (2015) · Zbl 1339.37089
[7] Benner, P.; Ohlberger, M.; Patera, A.; Rozza, G.; Urban, K., Model Reduction of Parametrized Systems (2017), Springer
[8] Billaud-Friess, M.; Nouy, A., Dynamical model reduction method for solving parameter-dependent dynamical systems, SIAM J. Sci. Comput., 39, 4, A1766-A1792 (2017) · Zbl 1373.65091
[9] Carlberg, K., Adaptive h-refinement for reduced-order models, Int. J. Numer. Methods Eng., 102, 5, 1192-1210 (2015) · Zbl 1352.65136
[10] Carlberg, K.; Bou-Mosleh, C.; Farhat, C., Efficient non-linear model reduction via a least-squares Petrov-Galerkin projection and compressive tensor approximations, Int. J. Numer. Methods Eng., 86, 2, 155-181 (2011) · Zbl 1235.74351
[11] Cazemier, W.; Verstappen, R. W.; Veldman, A. E., Proper orthogonal decomposition and low-dimensional models for driven cavity flows, Phys. Fluids (1998)
[12] Chaturantabut, S.; Sorensen, D. C., Nonlinear model reduction via discrete empirical interpolation, SIAM J. Sci. Comput. (2010) · Zbl 1217.65169
[13] Chinesta, F.; Ladeveze, P.; Cueto, E., A short review on model order reduction based on proper generalized decomposition, Arch. Comput. Methods Eng., 18, 4, 395-404 (2011)
[14] Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y., Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014), arXiv preprint
[15] Chollet, F., Keras (2015)
[16] Chorin, A.; Stinis, P., Problem reduction, renormalization, and memory, Commun. Appl. Math. Comput. Sci., 1, 1, 1-27 (2007) · Zbl 1108.82023
[17] Chorin, A. J.; Hald, O. H., Stochastic Tools in Mathematics and Science, vol. 3 (2009), Springer · Zbl 1184.60001
[18] Chorin, A. J.; Hald, O. H.; Kupferman, R., Optimal prediction and the Mori-Zwanzig representation of irreversible processes, Proc. Natl. Acad. Sci., 97, 7, 2968-2973 (2000) · Zbl 0968.60036
[19] Chorin, A. J.; Hald, O. H.; Kupferman, R., Optimal prediction with memory, Phys. D: Nonlinear Phenom., 166, 3-4, 239-257 (2002) · Zbl 1017.60046
[20] Everson, R.; Sirovich, L., Karhunen-Loève procedure for gappy data, J. Opt. Soc. Am. A (1995)
[21] Gallivan, K.; Vandendorpe, A.; Van Dooren, P., Model reduction via tangential interpolation (2002), Technical report · Zbl 1078.41016
[22] Givon, D.; Kupferman, R.; Hald, O. H., Existence proof for orthogonal dynamics and the Mori-Zwanzig formalism, Isr. J. Math., 145, 1, 221-241 (2005) · Zbl 1126.82018
[23] Glorot, X.; Bengio, Y., Understanding the difficulty of training deep feedforward neural networks, (Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (2010)), 249-256
[24] Goodfellow, I.; Bengio, Y.; Courville, A., Deep Learning (2016), MIT Press · Zbl 1373.68009
[25] Gouasmi, A.; Parish, E. J.; Duraisamy, K., A priori estimation of memory effects in reduced-order models of nonlinear systems using the Mori-Zwanzig formalism, Proc. R. Soc. A, Math. Phys. Eng. Sci., 473, 2205, Article 20170385 pp. (2017) · Zbl 1402.76065
[26] Halko, N.; Martinsson, P. G.; Tropp, J. A., Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions, SIAM Rev., 53, 217-288 (2011) · Zbl 1269.65043
[27] Hesthaven, J. S.; Rozza, G.; Stamm, B., Certified Reduced Basis Methods for Parametrized Partial Differential Equations (2015), Springer
[28] Hesthaven, J. S.; Warburton, T., Nodal Discontinuous Galerkin Methods, vol. 54 (2008), Springer
[29] Hochreiter, S.; Schmidhuber, J., Long short-term memory, Neural Comput., 9, 8, 1735-1780 (1997)
[30] Hsu, K.; Gupta, H. V.; Sorooshian, S., Artificial neural network modeling of the rainfall-runoff process, Water Resour. Res., 31, 10, 2517-2530 (1995)
[31] Huang, C.; Duraisamy, K.; Merkle, C., Challenges in reduced order modeling of reacting flows, (2018 Joint Propulsion Conference (2018)), 4675
[32] Ioffe, S.; Szegedy, C., Batch normalization: accelerating deep network training by reducing internal covariate shift (2015), arXiv preprint
[33] Iollo, A.; Lanteri, S.; Désidéri, J.-A., Stability properties of POD-Galerkin approximations for the compressible Navier-Stokes equations, Theor. Comput. Fluid Dyn., 13, 6, 377-396 (2000) · Zbl 0987.76077
[34] Karpathy, A.; Fei-Fei, L., Deep visual-semantic alignments for generating image descriptions, (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015))
[35] Kingma, D. P.; Ba Adam, J., Adam: a method for stochastic optimization (2014), arXiv preprint
[36] Kramer, B.; Willcox, K. E., Nonlinear model order reduction via lifting transformations and proper orthogonal decomposition, AIAA J., 57, 6, 2297-2307 (2019)
[37] Kutz, J. N.; Brunton, S. L.; Brunton, B. W.; Proctor, J. L., Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems (2016), SIAM · Zbl 1365.65009
[38] Lappano, E.; Naets, F.; Desmet, W.; Mundo, D.; Nijman, E., A greedy sampling approach for the projection basis construction in parametric model order reduction for structural dynamics models (2018), KU Leuven, Technical report
[39] Li, J.; Stinis, P., Mori-Zwanzig reduced models for uncertainty quantification, J. Comput. Dyn., 6, 1, 39-68 (2019) · Zbl 07100319
[40] Lu, F.; Lin, K. K.; Chorin, A. J., Data-based stochastic model reduction for the Kuramoto-Sivashinsky equation, Phys. D: Nonlinear Phenom., 340, 46-57 (2017) · Zbl 1376.35100
[41] Lucia, D. J.; Beran, P. S.; Silva, W. A., Reduced-order modeling: new approaches for computational physics, Prog. Aerosp. Sci., 40, 1-2, 51-117 (2004)
[42] Ma, C.; Wang, J.; E, W., Model reduction with memory and the machine learning of dynamical systems, Commun. Comput. Phys., 25, 4, 947-962 (2019)
[43] Maboudi Afkham, B.; Ripamonti, N.; Wang, Q.; Hesthaven, J. S., Conservative Model Order Reduction for Fluid Flow (2018), Ecole Polytechnique Fédérale de Lausanne, Technical report
[44] Maday, Y., Reduced basis method for the rapid and reliable solution of partial differential equations, (Proceedings of the International Congress of Mathematicians Madrid. Proceedings of the International Congress of Mathematicians Madrid, August 22-30, 2006 (2009), European Mathematical Society), 1255-1270 · Zbl 1100.65079
[45] Maulik, R.; Mohan, A.; Lusch, B.; Madireddy, S.; Balaprakash, P., Time-series learning of latent-space dynamics for reduced-order model closure (2019), arXiv preprint
[46] Mori, H., Transport, collective motion, and Brownian motion, Prog. Theor. Phys., 33, 3, 423-455 (1965) · Zbl 0127.45002
[47] Ohwada, T.; Asinari, P., Artificial compressibility method revisited: asymptotic numerical method for incompressible Navier-Stokes equations, J. Comput. Phys., 229, 5, 1698-1723 (2010) · Zbl 1329.76063
[48] Pan, S.; Duraisamy, K., Data-driven discovery of closure models, SIAM J. Appl. Dyn. Syst., 17, 4, 2381-2413 (2018) · Zbl 1411.70023
[49] Panzer, H.; Mohring, J., Parametric model order reduction by matrix interpolation, Automatisierungstechnik, 58, 8, 475-484 (2010)
[50] Parish, E. J.; Duraisamy, K., A dynamic subgrid scale model for large eddy simulations based on the Mori-Zwanzig formalism, J. Comput. Phys., 349, 154-175 (2017) · Zbl 1380.76024
[51] Parish, E. J.; Duraisamy, K., Non-Markovian closure models for large eddy simulations using the Mori-Zwanzig formalism, Phys. Rev. Fluids, 2, 1, Article 14604 pp. (2017)
[52] Parish, E. J.; Wentland, C.; Duraisamy, K., A residual-based Petrov-Galerkin reduced-order model with memory effects (2018), arXiv preprint
[53] Peherstorfer, B.; Willcox, K., Online adaptive model reduction for nonlinear systems via low-rank updates, SIAM J. Sci. Comput., 37, 4, A2123-A2150 (2015) · Zbl 1323.65102
[54] Quarteroni, A., Numerical Models for Differential Problems (2009), Springer Milan: Springer Milan Milano
[55] Quarteroni, A.; Formaggia, L., Mathematical modelling and numerical simulation of the cardiovascular system, Handb. Numer. Anal., 12, 3-127 (2004)
[56] Quarteroni, A.; Manzoni, A.; Negri, F., Reduced Basis Methods for Partial Differential Equations: An Introduction (2015), Springer
[57] Rohilla Shalizi, C.; Moore, C., What Is a Macrostate? Subjective Observations and Objective Dynamics (2000), Technical report
[58] Rowley, C. W.; Colonius, T.; Murray, R. M., Model reduction for compressible flows using POD and Galerkin projection, Phys. D: Nonlinear Phenom., 189, 1-2, 115-129 (2004) · Zbl 1098.76602
[59] Rozza, G.; Veroy, K., On the stability of the reduced basis method for Stokes equations in parametrized domains, Comput. Methods Appl. Mech. Eng., 196, 7, 1244-1260 (2007) · Zbl 1173.76352
[60] Stinis, P., Renormalized Mori-Zwanzig-reduced models for systems without scale separation, Proc. R. Soc. A, Math. Phys. Eng. Sci., 471, 2176, Article 20140446 pp. (2015) · Zbl 1371.82084
[61] Sutskever, I.; Vinyals, O.; Le, Q. V., Sequence to sequence learning with neural networks, (Advances in Neural Information Processing Systems (2014)), 3104-3112
[62] Unger, B.; Gugercin, S., Kolmogorov n-widths for linear dynamical systems, Adv. Comput. Math., 1-14 (May 2019)
[63] Vlachas, P. R.; Byeon, W.; Wan, Z. Y.; Sapsis, T. P.; Koumoutsakos, P., Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks, Proc. R. Soc. A, Math. Phys. Eng. Sci., 474, 2213, Article 20170844 pp. (2018) · Zbl 1402.92030
[64] Wan, Z. Y.; Vlachas, P.; Koumoutsakos, P.; Sapsis, T., Data-assisted reduced-order modeling of extreme events in complex dynamical systems, PLoS ONE, 13, 5, Article e0197704 pp. (2018)
[65] Wentland, C. R.; Huang, C.; Duraisamy, K., Closure of reacting flow reduced-order models via the adjoint Petrov-Galerkin method, (AIAA Aviation 2019 Forum (2019)), 3531
[66] Zhu, Y.; Dominy, J. M.; Venturi, D., On the estimation of the Mori-Zwanzig memory integral (2018) · Zbl 1402.60044
[67] Zwanzig, R., Memory effects in irreversible thermodynamics, Phys. Rev., 124, 4, 983-992 (1961) · Zbl 0131.45006
[68] Zwanzig, R., Nonlinear generalized Langevin equations, J. Stat. Phys., 9, 3, 215-220 (1973)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.