×

A multi-stage deep learning based algorithm for multiscale model reduction. (English) Zbl 1524.65522

Summary: In this work, we propose a multi-stage training strategy for the development of deep learning algorithms applied to problems with multiscale features. Each stage of the proposed strategy shares an (almost) identical network structure and predicts the same reduced order model of the multiscale problem. The output of the previous stage will be combined with an intermediate layer for the current stage. We numerically show that using different reduced order models as inputs of each stage can improve the training and we propose several ways of adding different information into the systems. These methods include mathematical multiscale model reductions and network approaches; but we found that the mathematical approach is a systematical way of decoupling information and gives the best result. We finally verified our training methodology on a time dependent nonlinear problem and a steady state model.

MSC:

65M60 Finite element, Rayleigh-Ritz and Galerkin methods for initial value and initial-boundary value problems involving PDEs
65N30 Finite element, Rayleigh-Ritz and Galerkin methods for boundary value problems involving PDEs
76S05 Flows in porous media; filtration; seepage
68T07 Artificial neural networks and deep learning
35R02 PDEs on graphs and networks (ramified or polygonal spaces)
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Brunton, Steven L.; Kutz, J. Nathan, Methods for data-driven multiscale model discovery for materials, J. Phys. Mater., 2, 4, Article 044002 pp. (2019)
[2] Cheung, Siu Wun; Chung, Eric T.; Efendiev, Yalchin; Gildin, Eduardo; Wang, Yating; Zhang, Jingyan, Deep global model reduction learning in porous media flow simulation, Comput. Geosci., 24, 1, 261-274 (2020) · Zbl 1434.76121
[3] Heinlein, Alexander; Klawonn, Axel; Lanser, Martin; Weber, Janine, Machine learning in adaptive domain decomposition methods—Predicting the geometric location of constraints, SIAM J. Sci. Comput., 41, 6, A3887-A3912 (2019) · Zbl 1429.65065
[4] Magiera, Jim; Ray, Deep; Hesthaven, Jan S.; Rohde, Christian, Constraint-aware neural networks for Riemann problems, J. Comput. Phys., 409, Article 109345 pp. (2020) · Zbl 1435.76046
[5] Regazzoni, F.; Dede, L.; Quarteroni, A., Machine learning of multiscale active force generation models for the efficient simulation of cardiac electromechanics, Comput. Methods Appl. Mech. Engrg., 370, Article 113268 pp. (2020) · Zbl 1506.74212
[6] Vasilyeva, Maria; Leung, Wing Tat; Chung, Eric T.; Efendiev, Yalchin; Wheeler, Mary, Learning macroscopic parameters in nonlinear multiscale simulations using nonlocal multicontinua upscaling techniques, J. Comput. Phys., Article 109323 pp. (2020) · Zbl 1436.76041
[7] Wang, Kun; Sun, WaiChing, A multiscale multi-permeability poroplasticity model linked by recursive homogenizations and deep learning, Comput. Methods Appl. Mech. Engrg., 334, 337-380 (2018) · Zbl 1440.74130
[8] Wang, Min; Cheung, Siu Wun; Leung, Wing Tat; Chung, Eric T.; Efendiev, Yalchin; Wheeler, Mary, Reduced-order deep learning for flow dynamics. The interplay between deep learning and model reduction, J. Comput. Phys., 401, Article 108939 pp. (2020) · Zbl 1454.76007
[9] Wang, Qian; Ripamonti, Nicolò; Hesthaven, Jan S., Recurrent neural network closure of parametric POD-Galerkin reduced-order models based on the Mori-Zwanzig formalism, J. Comput. Phys., Article 109402 pp. (2020) · Zbl 1436.65093
[10] Wang, Yating; Cheung, Siu Wun; Chung, Eric T.; Efendiev, Yalchin; Wang, Min, Deep multiscale model learning, J. Comput. Phys., 406, Article 109071 pp. (2020) · Zbl 1454.76008
[11] Yeung, Tak Shing Au; Chung, Eric T.; See, Simon, A deep learning based nonlinear upscaling method for transport equations (2020), arXiv preprint arXiv:2007.03432
[12] Zhang, Zecheng; Chung, Eric T.; Efendiev, Yalchin; Leung, Wing Tat, Learning algorithms for coarsening uncertainty space and applications to multiscale simulations, Mathematics, 8, 5, 720 (2020)
[13] Badrinarayanan, Vijay; Kendall, Alex; Cipolla, Roberto, Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 39, 12, 2481-2495 (2017)
[14] Dong, Chao; Loy, Chen Change; He, Kaiming; Tang, Xiaoou, Image super-resolution using deep convolutional networks, IEEE Trans. Pattern Anal. Mach. Intell., 38, 2, 295-307 (2015)
[15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
[16] Jie Hu, Li Shen, Gang Sun, Squeeze-and-excitation networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7132-7141.
[17] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, Kilian Q. Weinberger, Densely connected convolutional networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4700-4708.
[18] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A Efros, Image-to-image translation with conditional adversarial networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1125-1134.
[19] Johnson, Justin; Alahi, Alexandre; Li, Fei-Fei, Perceptual losses for real-time style transfer and super-resolution, (European Conference on Computer Vision (2016), Springer), 694-711
[20] Jonathan Long, Evan Shelhamer, Trevor Darrell, Fully convolutional networks for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3431-3440.
[21] Simonyan, Karen; Zisserman, Andrew, Very deep convolutional networks for large-scale image recognition (2014), arXiv preprint arXiv:1409.1556
[22] C. Szegedy, S. Ioffe, V. Vanhoucke, A. Alemi, Inception-ResNet and the impact of residual connections on learning, arXiv preprint arXiv:1602.07261.
[23] Szegedy, Christian; Ioffe, Sergey; Vanhoucke, Vincent; Alemi, Alexander A., Inception-v4, inception-resnet and the impact of residual connections on learning, (Thirty-First AAAI Conference on Artificial Intelligence (2017))
[24] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, Zbigniew Wojna, Rethinking the inception architecture for computer vision, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2818-2826.
[25] Calo, Victor M.; Efendiev, Yalchin; Galvis, Juan; Ghommem, Mehdi, Multiscale empirical interpolation for solving nonlinear PDEs, J. Comput. Phys., 278, 204-220 (2014) · Zbl 1349.65612
[26] Calo, Victor M.; Efendiev, Yalchin; Galvis, Juan; Li, Guanglian, Randomized oversampling for generalized multiscale finite element methods, Multiscale Model. Simul., 14, 1, 482-501 (2016) · Zbl 1337.65148
[27] Chung, Eric T.; Efendiev, Yalchin; Hou, Thomas Y., Adaptive multiscale model reduction with generalized multiscale finite element methods, J. Comput. Phys., 320, 69-95 (2016) · Zbl 1349.76191
[28] Chung, Eric T.; Efendiev, Yalchin, Reduced-contrast approximations for high-contrast multiscale flow problems, Multiscale Model. Simul., 8, 4, 1128-1153 (2010) · Zbl 1211.35073
[29] Chung, Eric T.; Efendiev, Yalchin; Leung, Wing Tat, Generalized multiscale finite element methods for wave propagation in heterogeneous media, Multiscale Model. Simul., 12, 4, 1691-1721 (2014) · Zbl 1315.65085
[30] Chung, Eric T.; Efendiev, Yalchin; Leung, Wing Tat, Residual-driven online generalized multiscale finite element methods, J. Comput. Phys., 302, 176-190 (2015) · Zbl 1349.65615
[31] Chung, Eric T.; Efendiev, Yalchin; Leung, Wing Tat, An online generalized multiscale discontinuous Galerkin method (GMsDGM) for flows in heterogeneous media, Commun. Comput. Phys., 21, 2, 401-422 (2017) · Zbl 1488.65609
[32] Chung, Eric T.; Efendiev, Yalchin; Leung, Wing Tat, Constraint energy minimizing generalized multiscale finite element method, Comput. Methods Appl. Mech. Engrg., 339, 298-319 (2018) · Zbl 1440.65195
[33] Chung, Eric T.; Efendiev, Yalchin; Li, Guanglian; Vasilyeva, Maria, Generalized multiscale finite element methods for problems in perforated heterogeneous domains, Appl. Anal., 95, 10, 2254-2279 (2016) · Zbl 1457.65189
[34] Chung, Eric T.; Leung, Wing Tat; Vasilyeva, Maria, Mixed GMsFEM for second order elliptic problem in perforated domains, J. Comput. Appl. Math., 304, 84-99 (2016) · Zbl 1382.65382
[35] Efendiev, Yalchin; Galvis, Juan; Hou, Thomas Y., Generalized multiscale finite element methods (GMsFEM), J. Comput. Phys., 251, 116-135 (2013) · Zbl 1349.65617
[36] Efendiev, Yalchin; Galvis, Juan; Lazarov, R.; Moon, M.; Sarkis, Marcus, Generalized multiscale finite element method. Symmetric interior penalty coupling, J. Comput. Phys., 255, 1-15 (2013) · Zbl 1349.76199
[37] Efendiev, Yalchin; Galvis, Juan; Li, Guanglian; Presho, Michael, Generalized multiscale finite element methods: Oversampling strategies, Int. J. Multiscale Comput. Eng., 12, 6 (2014) · Zbl 1388.65146
[38] Hou, Thomas Y.; Wu, Xiao-Hui, A multiscale finite element method for elliptic problems in composite materials and porous media, J. Comput. Phys., 134, 1, 169-189 (1997) · Zbl 0880.73065
[39] Goodfellow, Ian; Bengio, Yoshua; Courville, Aaron, Deep Learning (2016), MIT press · Zbl 1373.68009
[40] Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N.; Kaiser, Łukasz; Polosukhin, Illia, Attention is all you need, (Advances in Neural Information Processing Systems (2017)), 5998-6008
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.