zbMATH — the first resource for mathematics

Non-intrusive reduced order modeling of nonlinear problems using neural networks. (English) Zbl 1398.65330
Summary: We develop a non-intrusive reduced basis (RB) method for parametrized steady-state partial differential equations (PDEs). The method extracts a reduced basis from a collection of high-fidelity solutions via a proper orthogonal decomposition (POD) and employs artificial neural networks (ANNs), particularly multi-layer perceptrons (MLPs), to accurately approximate the coefficients of the reduced model. The search for the optimal number of neurons and the minimum amount of training samples to avoid overfitting is carried out in the offline phase through an automatic routine, relying upon a joint use of the Latin hypercube sampling (LHS) and the Levenberg-Marquardt (LM) training algorithm. This guarantees a complete offline-online decoupling, leading to an efficient RB method – referred to as POD-NN – suitable also for general nonlinear problems with a non-affine parametric dependence. Numerical studies are presented for the nonlinear Poisson equation and for driven cavity viscous flows, modeled through the steady incompressible Navier-Stokes equations. Both physical and geometrical parametrizations are considered. Several results confirm the accuracy of the POD-NN method and show the substantial speed-up enabled at the online stage as compared to a traditional RB strategy.

65N99 Numerical methods for partial differential equations, boundary value problems
68T05 Learning and adaptive systems in artificial intelligence
68W27 Online algorithms; streaming algorithms
35B30 Dependence of solutions to PDEs on initial and/or boundary data and/or on parameters of PDEs
76D05 Navier-Stokes equations for incompressible viscous fluids
PDF BibTeX Cite
Full Text: DOI
[1] Amsallem, D., Interpolation on manifolds of CFD-based fluid and finite element-based structural reduced-order models for on-line aeroelastic predictions, (2010), Department of Aeronautics and Astronautics, Stanford University, Doctoral dissertation
[2] Ballarin, F.; Manzoni, A.; Quarteroni, A.; Rozza, G., Supremizer stabilization of POD-Galerkin approximation of parametrized Navier-Stokes equations, (2014), École Polytechnique Fédérale de Lausanne, MATHICSE technical report
[3] Barrault, M.; Maday, Y.; Nguyen, N. C.; Patera, A. T., An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations, C. R. Math., 339, 9, 667-672, (2004) · Zbl 1061.65118
[4] Barthelmann, V.; Novak, E.; Ritter, K., High-dimensional polynomial interpolation on sparse grids, Adv. Comput. Math., 12, 4, 273-288, (2000) · Zbl 0944.41001
[5] Bendsøe, M. P.; Sigmund, O., Topology optimization: theory, methods and applications, (2004), Springer Science & Business Media Heidelberg · Zbl 1059.74001
[6] Brown, P. F.; Pietra, V. J.D.; Pietra, S. A.D.; Mercer, R. L., The mathematics of statistical machine translation: parameter estimation, Comput. Linguist., 19, 2, 263-311, (1993)
[7] Buffa, A.; Maday, Y.; Patera, A. T.; Prud’Homme, C.; Turinici, G., A priori convergence of the greedy algorithm for the parametrized reduced basis method, ESAIM: Math. Model. Numer. Anal., 46, 595-603, (2012) · Zbl 1272.65084
[8] Burkardt, J.; Gunzburger, M.; Lee, H. C., POD and CVT-based reduced-order modeling of Navier-Stokes flows, Comput. Methods Appl. Mech. Eng., 196, 337-355, (2006) · Zbl 1120.76323
[9] Casenave, F.; Ern, A.; Lelièvre, T., A nonintrusive reduced basis method applied to aeroacoustic simulations, Adv. Comput. Math., 41, 961-986, (2015) · Zbl 1338.76049
[10] Chaturantabut, S.; Sorensen, D. C., Nonlinear model reduction via discrete empirical interpolation, SIAM J. Sci. Comput., 32, 5, 2737-2764, (2010) · Zbl 1217.65169
[11] Chen, W.; Hesthaven, J. S.; Junqiang, B.; Yang, Z.; Tihao, Y., A greedy non-intrusive reduced order model for fluid dynamics, J. Northwest. Polytech. Univ., (2017)
[12] Cybenko, G., Continuous valued neural networks with two hidden layers are sufficient, (1988), Department of Computer Science, Tufts University, Technical report
[13] Cybenko, G., Approximation by superpositions of a sigmoidal function, Math. Control Signals Syst., 2, 4, 303-314, (1989) · Zbl 0679.94019
[14] De Boor, C., A practical guide to splines, (1978), Springer-Verlag New York, NY · Zbl 0406.41003
[15] Deparis, S., Reduced basis error bound computation of parameter-dependent Navier-Stokes equations by the natural norm approach, SIAM J. Numer. Anal., 46, 4, 2039-2067, (2008) · Zbl 1177.35148
[16] Dhondt, G., Calculix crunchix User’s manual, (2014), available at
[17] Eckart, C.; Young, G., The approximation of one matrix by another of lower rank, Psychometrika, 1, 211-218, (1936) · JFM 62.1075.02
[18] Eftang, J. L., Reduced basis methods for partial differential equations, (2008), Department of Mathematical Sciences, Norwegian University of Science and Technology, Master thesis
[19] Gill, P. E.; Murray, W.; Wright, M. H., Practical optimization, (1981), Academic Press · Zbl 0503.90062
[20] Hagan, M. T.; Menhaj, M. B., Training feedforward networks with the Marquardt algorithm, IEEE Trans. Neural Netw., 5, 6, 989-993, (1994)
[21] Hagan, M. T.; Demuth, H. B.; Beale, M. H., Neural network design, (1996), PWS Publishing Boston, MA
[22] Haykin, S., Neural networks: A comprehensive foundation, (2004), Prentice Hall Upper Saddle River, NJ · Zbl 0828.68103
[23] Hesthaven, J. S.; Stamn, B.; Rozza, G., Certified reduced basis methods for parametrized partial differential equations, (2016), Springer New York, NY · Zbl 1329.65203
[24] Hesthaven, J. S.; Stamn, B.; Zhang, S., Efficienty greedy algorithms for high-dimensional parameter spaces with applications to empirical interpolation and reduced basis methods, ESAIM: Math. Model. Numer. Anal., 48, 1, 259-283, (2014) · Zbl 1292.41001
[25] Imam, R. L., Latin hypercube sampling, (Encyclopedia of Quantitative Risk Analysis and Assessment, (2008))
[26] Jaggli, C.; Iapichino, L.; Rozza, G., An improvement on geometrical parametrizations by transfinite maps, C. R. Acad. Sci. Paris, Ser. I, 352, 263-268, (2014) · Zbl 1302.35167
[27] Kohavi, R., A study of cross-validation and bootstrap for accuracy estimation and model selection, (Proceedings of the 40th International Joint Conference on Artificial Intelligence, vol. 2, (1995)), 1137-1143
[28] Kriesel, D., A brief introduction to neural networks, (2007), retrieved from
[29] Le Maître, O.; Knio, O. M., Spectral methods for uncertainty quantification with applications to computational fluid dynamics, (2010), Springer Science & Business Media Berlin · Zbl 1193.76003
[30] Levenberg, K., A method for the solution of certain non-linear problems in least squares, Q. Appl. Math., 2, 164-168, (1944) · Zbl 0063.03501
[31] Lee, E. B.; Markus, L., Foundations of optimal control theory, (1967), John Wiley & Sons New York, NY · Zbl 0159.13201
[32] Liang, Y. C.; Lee, H. P.; Lim, S. P.; Lin, W. Z.; Lee, K. H.; Wu, C. G., Proper orthogonal decomposition and its applications, part I: theory, J. Sound Vib., 252, 3, 527-544, (2002) · Zbl 1237.65040
[33] Maday, Y., Reduced basis method for the rapid and reliable solution of partial differential equations, (Proceedings of the International Congress of Mathematicians, Madrid, Spain, (2006)), 1255-1269 · Zbl 1100.65079
[34] Marquardt, D. W., An algorithm for least-squares estimation of nonlinear parameters, J. Soc. Ind. Appl. Math., 11, 2, 431-441, (1963) · Zbl 0112.10505
[35] The MathWorks, Inc, Machine learning challenges: choosing the best model and avoiding overfitting, (2016), retrieved from
[36] Mitchell, W.; McClain, M. A., A collection of 2D elliptic problems for testing adaptive algorithms, (NISTIR, (2010)), 7668
[37] Manzoni, A.; Negri, F., Automatic reduction of PDEs defined on domains with variable shape, (2016), École Polytechnique Fédérale de Lausanne, MATHICSE technical report
[38] Møller, M. D., A scaled conjugate gradient algorithm for fast supervised learning, Neural Netw., 6, 525-533, (1993)
[39] Nielsen, M. A., Neural networks and deep learning, (2015), Determination Press
[40] Negri, F.; Manzoni, A.; Amsallem, D., Efficient model reduction of parametrized systems by matrix discrete empirical interpolation, J. Comput. Phys., 303, 431-454, (2015) · Zbl 1349.65154
[41] Persson, P. O., Implementation of finite-element based Navier-Stokes solver, (2002), Massachussets Institue of Technology
[42] Prud’homme, C.; Rovas, D. V.; Veroy, K.; Machiels, L.; Maday, Y.; Patera, A. T.; Turinici, G., Reliable real-time solution of parametrized partial differential equations: reduced-basis output bound methods, J. Fluids Eng., 124, 1, 70-80, (2002)
[43] Quarteroni, A., Numerical models for differential problems, vol. 2, (2010), Springer Science & Business Media New York, NY
[44] Quarteroni, A.; Manzoni, A.; Negri, F., Reduced basis methods for partial differential equations: an introduction, (2015), Springer New York, NY, 2015
[45] Rannacher, R., Finite element methods for the incompressible Navier-Stokes equations, Lecture Notes, (1999), Institute of Applied Mathematics, University of Heidelberg
[46] Rosenblatt, F., The perceptron: a probabilistic model for information storage and organization in the brain, Psychol. Rev., 65, 386-408, (1958)
[47] Schmidt, E., Zur theorie der linearen und nichtlinearen integralgleichungen. I. teil: entwicklung willkürlicher funktionen nach systemen vorgeschriebener, Math. Ann., 63, 433-476, (1907) · JFM 38.0377.02
[48] Stergiou, C.; Siganos, D., Neural networks, (2013), retrieved from
[49] Volkwein, S., Model reduction using proper orthogonal decomposition, Lecture Notes, (2008), Institute of Applied Mathematics, University of Konstanz · Zbl 1191.49040
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.