×

Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning. (English) Zbl 1452.68147

Summary: Data I/O poses a significant bottleneck in large-scale CFD simulations; thus, practitioners would like to significantly reduce the number of times the solution is saved to disk, yet retain the ability to recover any field quantity (at any time instance) a posteriori. The objective of this work is therefore to accurately recover missing CFD data a posteriori at any time instance, given that the solution has been written to disk at only a relatively small number of time instances. We consider in particular high-order discretizations (e.g., discontinuous Galerkin), as such techniques are becoming increasingly popular for the simulation of highly separated flows. To satisfy this objective, this work proposes a methodology consisting of two stages: 1) dimensionality reduction and 2) dynamics learning. For dimensionality reduction, we propose a novel hierarchical approach. First, the method reduces the number of degrees of freedom within each element of the high-order discretization by applying autoencoders from deep learning. Second, the methodology applies principal component analysis to compress the global vector of encodings. This leads to a low-dimensional state, which associates with a nonlinear embedding of the original CFD data. For dynamics learning, we propose to apply regression techniques (e.g., kernel methods) to learn the discrete-time velocity characterizing the time evolution of this low-dimensional state. A numerical example on a large-scale CFD example characterized by nearly 13 million degrees of freedom illustrates the suitability of the proposed method in an industrial setting.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
68T07 Artificial neural networks and deep learning
76M10 Finite element methods applied to problems in fluid mechanics

Software:

LIBSVM; PyFR; Adam; darch
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Altman, N. S., An introduction to kernel and nearest-neighbor nonparametric regression, Am. Stat., 46, 175-185 (1992)
[2] Beckers, J.-M.; Rixen, M., EOF calculations and data filling from incomplete oceanographic datasets, J. Atmos. Ocean. Technol., 20, 1839-1856 (2003)
[3] Böhmer, W.; Springenberg, J. T.; Boedecker, J.; Riedmiller, M.; Obermayer, K., Autonomous learning of state representations for control: an emerging field aims to autonomously learn state representations for reinforcement learning agents from their real-world sensor observations, KI-Künstl. Intell., 29, 353-362 (2015)
[4] Bottou, L.; Curtis, F. E.; Nocedal, J., Optimization methods for large-scale machine learning, SIAM Rev., 60, 223-311 (2018) · Zbl 1397.65085
[5] Breiman, L., Random forests, Mach. Learn., 45, 5-32 (2001) · Zbl 1007.68152
[6] Brunton, S. L.; Proctor, J. L.; Kutz, J. N., Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci., Article 201517384 pp. (2016) · Zbl 1355.94013
[7] Bui-Thanh, T.; Damodaran, M.; Willcox, K., Aerodynamic data reconstruction and inverse design using proper orthogonal decomposition, AIAA J., 42, 1505-1516 (2004)
[8] Carlberg, K.; Barone, M.; Antil, H., Galerkin v. least-squares Petrov-Galerkin projection in nonlinear model reduction, J. Comput. Phys., 330, 693-734 (2017) · Zbl 1378.65145
[9] Chang, C. Chung; Lin, C. Jen, LIBSVM: a library for support vector machines, ACM Trans. Intell. Syst. Technol., 2 (2001)
[10] Cockburn, B.; Shu, C.-W., The Runge-Kutta local projection P1-discontinuous-Galerkin finite element method for scalar conservation laws, ESAIM: Math. Model. Numer. Anal., 25, 337-361 (1991) · Zbl 0732.65094
[11] Goroshin, R.; Mathieu, M. F.; LeCun, Y., Learning to linearize under uncertainty, (Cortes, C.; Lawrence, N. D.; Lee, D. D.; Sugiyama, M.; Garnett, R., Advances in Neural Information Processing Systems 28 (2015), Curran Associates, Inc.), 1234-1242
[12] Gunes, H.; Sirisup, S.; Karniadakis, G. E., Gappy data: to krig or not to krig?, J. Comput. Phys., 212, 358-382 (2006) · Zbl 1216.76062
[13] Hastie, T.; Tibshirani, R.; Friedman, J. H., The Elements of Statistical Learning (2009), Springer · Zbl 1273.62005
[14] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, 2016.; K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, 2016.
[15] Hesthaven, J. S.; Warburton, T., Nodal Discontinuous Galerkin Methods: Algorithms, Analysis, and Applications, vol. 54 (2008), Springer Verlag: Springer Verlag New York · Zbl 1134.65068
[16] Hinton, G. E.; Salakhutdinov, R. R., Reducing the dimensionality of data with neural networks, Science, 313, 504-507 (2006) · Zbl 1226.68083
[17] Holmes, P.; Lumley, J.; Berkooz, G., Turbulence, Coherent Structures, Dynamical Systems and Symmetry (1996), Cambridge University Press · Zbl 0890.76001
[18] Huynh, H. T., A flux reconstruction approach to high-order schemes including discontinuous Galerkin methods, (18th AIAA Computational Fluid Dynamics Conference (2007)), 4079
[19] Ioffe, S.; Szegedy, C., Batch normalization: accelerating deep network training by reducing internal covariate shift (2015), arXiv preprint
[20] Kang, H.; Lee, D.; Lee, D., A study on CFD data compression using hybrid supercompact wavelets, KSME Int. J., 17, 1784-1792 (2003)
[21] Karl, M.; Soelch, M.; Bayer, J.; van der Smagt, P., Deep variational Bayes filters: unsupervised learning of state space models from raw data (2016), arXiv preprint
[22] Kawahara, Y., Dynamic mode decomposition with reproducing kernels for Koopman spectral analysis, (Advances in Neural Information Processing Systems (2016)), 911-919
[23] Kennedy, C. A.; Carpenter, M. H.; Lewis, R. M., Low-storage, explicit Runge-Kutta schemes for the compressible Navier-Stokes equations, Appl. Numer. Math., 35, 177-219 (1999) · Zbl 0986.76060
[24] Kingma, D.; Ba, J., Adam: a method for stochastic optimization (2014), arXiv preprint
[25] Kingma, D. P.; Welling, M., Auto-encoding variational Bayes (2013), arXiv preprint
[26] Kopriva, D. A., A staggered-grid multidomain spectral method for the compressible Navier-Stokes equations, J. Comput. Phys., 143, 125-158 (1998) · Zbl 0921.76121
[27] Kumarasamy, S.; Barlow, J. B., Computation of unsteady flow over a half-cylinder close to a moving wall, J. Wind Eng. Ind. Aerodyn., 69, 239-248 (1997)
[28] Lesort, T.; Díaz-Rodríguez, N.; Goudou, J.-F.; Filliat, D., State representation learning for control: an overview, Neural Netw. (2018)
[29] Liu, Y.; Vinokur, M.; Wang, Z., Spectral difference method for unstructured grids I: basic formulation, J. Comput. Phys., 216, 780-801 (2006) · Zbl 1097.65089
[30] Lusch, B.; Kutz, J. N.; Brunton, S. L., Deep learning for universal linear embeddings of nonlinear dynamics (2017), arXiv preprint
[31] J. Morton, F.D. Witherden, A. Jameson, M.J. Kochenderfer, Deep dynamical modeling and control of unsteady fluid flows, 2018.; J. Morton, F.D. Witherden, A. Jameson, M.J. Kochenderfer, Deep dynamical modeling and control of unsteady fluid flows, 2018.
[32] Nair, V.; Hinton, G. E., Rectified linear units improve restricted Boltzmann machines, (Proceedings of the 27th International Conference on Machine Learning. Proceedings of the 27th International Conference on Machine Learning, ICML-10 (2010)), 807-814
[33] Nakamura, Y., Vortex shedding from bluff bodies with splitter plates, J. Fluids Struct., 10, 147-158 (1996)
[34] Otto, S. E.; Rowley, C. W., Linearly-recurrent autoencoder networks for learning dynamics (2017), arXiv preprint · Zbl 1489.65164
[35] Park, J.; Witherden, F.; Vincent, P., High-Order Implicit Large-Eddy Simulations of Flow Over a NACA0021 Aerofoil, 2186-2197 (2017), AIAA Journal
[36] Parnaudeau, P.; Carlier, J.; Heitz, D.; Lamballais, E., Experimental and numerical studies of the flow over a circular cylinder at Reynolds number 3900, Phys. Fluids, 20, Article 085101 pp. (2008) · Zbl 1182.76591
[37] Puligilla, S. C.; Jayaraman, B., Deep multilayer convolution frameworks for data-driven learning of fluid flow dynamics, (2018 Fluid Dynamics Conference (2018)), 3091
[38] Reed, W. H.; Hill, T. R., Triangular Mesh Methods for the Neutron Transport Equation (1973), Los Alamos Scientific Laboratory, Technical Report LA-UR-73-479
[39] Sakai, R.; Sasaki, D.; Nakahashi, K., Parallel implementation of large-scale CFD data compression toward aeroacoustic analysis, Comput. Fluids, 80, 116-127 (2013)
[40] Santa Cruz, A.; David, L.; Pecheux, J.; Texier, A., Characterization by proper-orthogonal-decomposition of the passive controlled wake flow downstream of a half cylinder, Exp. Fluids, 39, 730-742 (2005)
[41] Schmid, P. J., Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656, 5-28 (2010) · Zbl 1197.76091
[42] Smola, A. J.; Schölkopf, B., A tutorial on support vector regression, Stat. Comput., 14, 199-222 (2004)
[43] Sun, Y.; Wang, Z. J.; Liu, Y., High-order multidomain spectral difference method for the Navier-Stokes equations on unstructured hexahedral grids, Commun. Comput. Phys., 2, 310-333 (2007) · Zbl 1164.76360
[44] Takeishi, N.; Kawahara, Y.; Yairi, T., Learning Koopman invariant subspaces for dynamic mode decomposition, (Advances in Neural Information Processing Systems (2017)), 1130-1140
[45] Theis, L.; Shi, W.; Cunningham, A.; Huszár, F., Lossy image compression with compressive autoencoders, (International Conference on Learning Representations (2017))
[46] Tibshirani, R., Regression shrinkage and selection via the lasso, J. R. Stat. Soc., Ser. B, Methodol., 267-288 (1996) · Zbl 0850.62538
[47] Trott, A.; Moorhead, R.; McGinley, J., Wavelets applied to loseless compression and progressive transmission of floating point data in 3-d curvilinear grids, (Proceedings of the 7th Conference on Visualization’96 (1996), IEEE Computer Society Press), 385
[48] Venturi, D.; Karniadakis, G. E., Gappy data and reconstruction procedures for flow past a cylinder, J. Fluid Mech., 519, 315-336 (2004) · Zbl 1065.76159
[49] Vermeire, B. C.; Witherden, F. D.; Vincent, P. E., On the utility of GPU accelerated high-order methods for unsteady flow simulations: a comparison with industry-standard tools, J. Comput. Phys., 334, 497-521 (2017)
[50] Vincent, P.; Larochelle, H.; Lajoie, I.; Bengio, Y.; Manzagol, P.-A., Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion, 11, 3371-3408 (2010) · Zbl 1242.68256
[51] Vincent, P.; Witherden, F.; Vermeire, B.; Park, J. S.; Iyer, A., Towards green aviation with Python at petascale, (Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (2016), IEEE Press), 1
[52] Vincent, P. E.; Jameson, A., Facilitating the adoption of unstructured high-order methods amongst a wider community of fluid dynamicists, Math. Model. Nat. Phenom., 6, 97-140 (2011) · Zbl 1387.76002
[53] Watter, M.; Springenberg, J.; Boedecker, J.; Riedmiller, M., Embed to control: a locally linear latent dynamics model for control from raw images, (Advances in Neural Information Processing Systems (2015)), 2746-2754
[54] Wiewel, S.; Becher, M.; Thuerey, N., Latent-space physics: towards learning the temporal evolution of fluid flow (2018), arXiv preprint
[55] Willcox, K., Unsteady flow sensing and estimation via the gappy proper orthogonal decomposition, Comput. Fluids, 35, 208-226 (2006) · Zbl 1160.76394
[56] Williams, M. O.; Kevrekidis, I. G.; Rowley, C. W., A data – driven approximation of the Koopman operator: extending dynamic mode decomposition, J. Nonlinear Sci., 25, 1307-1346 (2015) · Zbl 1329.65310
[57] Williams, M. O.; Rowley, C. W.; Kevrekidis, I. G., A kernel-based approach to data-driven Koopman spectral analysis (2014), arXiv preprint
[58] Wirtz, D.; Haasdonk, B., A vectorial kernel orthogonal greedy algorithm, Dolomites Res. Notes Approx., 6 (2013)
[59] Wirtz, D.; Karajan, N.; Haasdonk, B., Surrogate modeling of multiscale models using kernel methods, Int. J. Numer. Methods Eng., 101, 1-28 (2015) · Zbl 1352.65144
[60] Witherden, F. D.; Vermeire, B. C.; Vincent, P. E., Heterogeneous computing on mixed unstructured grids with PyFR, Comput. Fluids, 120, 173-186 (2015) · Zbl 1390.76014
[61] Zou, H.; Hastie, T., Regularization and variable selection via the elastic net, J. R. Stat. Soc., Ser. B, Stat. Methodol., 67, 301-320 (2005) · Zbl 1069.62054
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.