×

Toward extremely scalable nonlinear domain decomposition methods for elliptic partial differential equations. (English) Zbl 1329.65294

Summary: The solution of nonlinear problems, e.g., in material science, requires fast and highly scalable parallel solvers. Finite element tearing and interconnecting dual primal (FETI-DP) domain decomposition methods are parallel solution methods for implicit problems discretized by finite elements. Recently, nonlinear versions of the well-known FETI-DP methods for linear problems have been introduced. In these methods, the nonlinear problem is decomposed before linearization. This approach can be viewed as a strategy to further localize computational work and to extend the parallel scalability of FETI-DP methods toward extreme-scale supercomputers. Here, a recent nonlinear FETI-DP method is combined with an approach that allows an inexact solution of the FETI-DP coarse problem. We combine the nonlinear FETI-DP domain decomposition method with an algebraic multigrid (AMG) method and thus obtain a hybrid nonlinear domain decomposition/multigrid method. We consider scalar nonlinear problems as well as nonlinear hyperelasticity problems in two and three space dimensions. For the first time for a domain decomposition method, weak parallel scalability can be shown beyond half a million cores and subdomains. We can show weak parallel scalability for up to \(524288\) cores on the Mira Blue Gene/Q supercomputer for our new implementation and discuss the steps necessary to obtain these results. We solve a heterogeneous nonlinear hyperelasticity problem discretized using piecewise quadratic finite elements with a total of \(42\) billion degrees of freedom in about six minutes. Our analysis reveals that scalability beyond \(524288\) cores depends critically on both efficient construction and solution of the coarse problem.

MSC:

65N55 Multigrid methods; domain decomposition for boundary value problems involving PDEs
65N30 Finite element, Rayleigh-Ritz and Galerkin methods for boundary value problems involving PDEs
35J60 Nonlinear elliptic equations
65Y05 Parallel numerical computation
74B20 Nonlinear elasticity
74S05 Finite element methods applied to problems in solid mechanics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] A. H. Baker, R. D. Falgout, T. V. Kolev, and U. M. Yang, {\it Scaling hypre’s multigrid solvers to 100,000 cores}, in High-Performance Scientific Computing, M. W. Berry, K. A. Gallivan, E. Gallopoulos, A. Grama, B. Philippe, Y. Saad, and F. Saied, eds., Springer, London, 2012, pp. 261-279.
[2] S. Balay, J. Brown, K. Buschelman, V. Eijkhout, W. D. Gropp, D. Kaushik, M. G. Knepley, L. C. McInnes, B. F. Smith, and H. Zhang, {\it PETSc Users Manual}, Technical Report ANL-95/11, Revision 3.5, Argonne National Laboratory, Lemont, IL, 2014.
[3] S. Balay, J. Brown, K. Buschelman, W. D. Gropp, D. Kaushik, M. G. Knepley, L. C. McInnes, B. F. Smith, and H. Zhang, {\it PETSc Web Page}, ŭlhttp://www.mcs.anl.gov/petsc (2014).
[4] S. Balay, W. D. Gropp, L. C. McInnes, and B. F. Smith, {\it Efficient management of parallelism in object oriented numerical software libraries}, in Modern Software Tools in Scientific Computing, E. Arge, A. M. Bruaset, and H. P. Langtangen, eds., Birkhäuser, Basel, 1997, pp. 163-202. · Zbl 0882.65154
[5] P. Bastian, M. Blatt, A. Dedner, C. Engwer, R. Klöfkorn, M. Ohlberger, and O. Sander, {\it A generic grid interface for parallel and adaptive scientific computing. Part I: Abstract framework}, Computing, 82 (2008), pp. 103-119. · Zbl 1151.65089
[6] P. Bastian, M. Blatt, A. Dedner, C. Engwer, R. Klöfkorn, M. Ohlberger, and O. Sander, {\it A generic grid interface for parallel and adaptive scientific computing. Part II: Implementation and tests in dune}, Computing, 82 (2008), pp. 121-138. · Zbl 1151.65088
[7] M. Bhardwaj, K. H. Pierson, G. Reese, T. Walsh, D. Day, K. Alvin, J. Peery, C. Farhat, and M. Lesoinne, {\it Salinas: A scalable software for high performance structural and mechanics simulation}, in ACM/IEEE Proceedings of SC02: High Performance Networking and Computing, Gordon Bell Award, ACM, New York, IEEE, Washington, DC, 2002, pp. 1-19.
[8] F. Bordeu, P.-A. Boucard, and P. Gosselet, {\it Balancing domain decomposition with nonlinear relocalization: Parallel implementation for laminates}, in Proceedings of the 1st International Conference on Parallel, Distributed and Grid Computing for Engineering, B. H. V. Topping and P. Ivnyi, eds., Civil-Comp Press, Stirlingshire, UK, 2009.
[9] P. Brune, M. G. Knepley, B. Smith, and X. Tu, {\it Composing Scalable Nonlinear Algebraic Solvers}, Technical Report ANL/MCS-P2010-0112, Argonne National Laboratory, Lemont, IL, 2013. · Zbl 1336.65030
[10] X.-C. Cai and M. Dryja, {\it Domain decomposition methods for monotone nonlinear elliptic problems}, in Domain Decomposition Methods in Scientific and Engineering Computing (University Park, PA, 1993), Contemp. Math. 180, AMS, Providence, RI, 1994, pp. 21-27. · Zbl 0817.65127
[11] X.-C. Cai and D. E. Keyes, {\it Nonlinearly preconditioned inexact Newton algorithms}, SIAM J. Sci. Comput., 24 (2002), pp. 183-200. · Zbl 1015.65058
[12] X.-C. Cai, D. E. Keyes, and L. Marcinkowski, {\it Non-linear additive Schwarz preconditioners and application in computational fluid dynamics}, Internat. J. Numer. Methods Fluids, 40 (2002), pp. 1463-1470. · Zbl 1025.76040
[13] P. Cresta, O. Allix, C. Rey, and S. Guinard, {\it Nonlinear localization strategies for domain decomposition methods: Application to post-buckling analyses}, Comput. Methods Appl. Mech. Engrg., 196 (2007), pp. 1436-1446. · Zbl 1173.74408
[14] J.-M. Cros, {\it A preconditioner for the Schur complement domain decomposition method}, in Domain Decomposition Methods in Science and Engineering, O. Widlund, I. Herrera, D. Keyes, and R. Yates, eds., National Autonomous University of Mexico (UNAM), Mexico City, Mexico, 2003, pp. 373-380.
[15] T. A. Davis, {\it A column pre-ordering strategy for the unsymmetric-pattern multifrontal method}, ACM Trans. Math. Software, 30 (2004), pp. 165-195. · Zbl 1072.65036
[16] H. De Sterck, U. M. Yang, and J. J. Heys, {\it Reducing complexity in parallel algebraic multigrid preconditioners}, SIAM J. Matrix Anal. Appl., 27 (2006), pp. 1019-1039. · Zbl 1102.65034
[17] S. Deparis, {\it Numerical Analysis of Axisymmetric Flows and Methods for Fluid-Structure Interaction Arising in Blood Flow Simulation}, Ph.D. thesis, EPFL, Lausanne, Switzerland, 2004.
[18] S. Deparis, M. Discacciati, G. Fourestey, and A. Quarteroni, {\it Heterogeneous domain decomposition methods for fluid-structure interaction problems}, in Domain Decomposition Methods in Science and Engineering XVI, Lect. Notes Comput. Sci. Eng. 55, Springer, Berlin, Heidelberg, 2007, pp. 41-52.
[19] S. Deparis, M. Discacciati, G. Fourestey, and A. Quarteroni, {\it Fluid-structure algorithms based on Steklov-Poincaré operators}, Comput. Methods Appl. Mech. Engrg., 195 (2006), pp. 5797-5812. · Zbl 1124.76026
[20] C. R. Dohrmann, {\it A preconditioner for substructuring based on constrained energy minimization}, SIAM J. Sci. Comput., 25 (2003), pp. 246-258. · Zbl 1038.65039
[21] M. Dryja and W. Hackbusch, {\it On the nonlinear domain decomposition method}, BIT, 37 (1997), pp. 296-311. · Zbl 0891.65126
[22] R. D. Falgout, J. E. Jones, and U. M. Yang, {\it The design and implementation of hypre, a library of parallel high performance preconditioners}, in Numerical Solution of Partial Differential Equations on Parallel Computers, A. M. Bruaset, P. Bjorstad, and A. Tveito, eds., Lect. Notes Comput. Sci. Engrg. 51, Springer-Verlag, Berlin, 2006, pp. 267-294. · Zbl 1097.65059
[23] C. Farhat, M. Lesoinne, P. LeTallec, K. Pierson, and D. Rixen, {\it FETI-DP: A dual-primal unified FETI method - part I: A faster alternative to the two-level FETI method}, Internat. J. Numer. Methods Engrg., 50 (2001), pp. 1523-1544. · Zbl 1008.74076
[24] C. Farhat, M. Lesoinne, and K. Pierson, {\it A scalable dual-primal domain decomposition method}, Numer. Linear Algebra Appl., 7 (2000), pp. 687-714. · Zbl 1051.65119
[25] M. Á. Fernández, J.-F. Gerbeau, A. Gloria, and M. Vidrascu, {\it Domain decomposition based Newton methods for fluid-structure interaction problems}, in CANUM 2006–Congrès National d’Analyse Numérique, ESAIM Proc. 22, EDP Sci., Les Ulis, 2008, pp. 67-82. · Zbl 1133.74043
[26] B. Ganis, K. Kumar, G. Pencheva, M. F. Wheeler, and I. Yotov, {\it A global Jacobian method for mortar discretizations of a fully implicit two-phase flow model}, Multiscale Model. Simul., 12 (2014), pp. 1401-1423. · Zbl 1312.76027
[27] B. Ganis, G. Pencheva, M. F. Wheeler, T. Wildey, and I. Yotov, {\it A frozen Jacobian multiscale mortar preconditioner for nonlinear interface operators}, Multiscale Model. Simul., 10 (2012), pp. 853-873. · Zbl 1255.76132
[28] A. Greenbaum, {\it Iterative Methods for Solving Linear Systems}, Frontiers Appl. Math. 17, SIAM, Philadelphia, 1997. · Zbl 0883.65022
[29] C. Groß and R. Krause, {\it A Generalized Recursive Trust-Region Approach–Nonlinear Multiplicatively Preconditioned Trust-Region Methods and Applications}, Technical Report 2010-09, Institute of Computational Science, Universita della Svizzera Italiana, Lugano, Switzerland, 2010.
[30] C. Groß and R. Krause, {\it On the Globalization of ASPIN Employing Trust-Region Control Strategies–Convergence Analysis and Numerical Examples}, Technical Report 2011-03, Institute of Computational Science, Universita della Svizzera Italiana, Lugano, Switzerland, 2011.
[31] V. E. Henson and U. M. Yang, {\it BoomerAMG: A parallel algebraic multigrid solver and preconditioner}, Appl. Numer. Math., 41 (2002), pp. 155-177. · Zbl 0995.65128
[32] G. A. Holzapfel, {\it Nonlinear Solid Mechanics. A Continuum Approach for Engineering}, John Wiley and Sons, Chichester, UK, 2000. · Zbl 0980.74001
[33] F.-N. Hwang and X.-C. Cai, {\it Improving robustness and parallel scalability of Newton method through nonlinear preconditioning}, in Domain Decomposition Methods in Science and Engineering, Lect. Notes Comput. Sci. Eng. 40, Springer, Berlin, 2005, pp. 201-208. · Zbl 1067.65052
[34] F.-N. Hwang and X.-C. Cai, {\it A class of parallel two-level nonlinear Schwarz preconditioned inexact Newton algorithms}, Comput. Methods Appl. Mech. Engrg., 196 (2007), pp. 1603-1611. · Zbl 1173.76385
[35] O. Ippisch, M. Blatt, J. Fahlke, and F. Heimann, {\it MuPhi–Simulation of Flow and Transport in Porous Media}, http://www.fz-juelich.de/ias/jsc/EN/Expertise/High-Q-Club/muPhi/_node.html (2014).
[36] P. Jolivet, F. Hecht, F. Nataf, and C. Prud’homme, {\it Scalable domain decomposition preconditioners}, in SuperComputing SC13, Denver, CO, 2013.
[37] A. Klawonn, M. Lanser, P. Radtke, and O. Rheinbach, {\it On an adaptive coarse space and on nonlinear domain decomposition}, in Domain Decomposition Methods in Science and Engineering XXI, Lect. Notes Comput. Sci. Eng. 98, J. Erhel, M. J. Gander, L. Halpern, G. Pichot, T. Sassi, and O. B. Widlund, eds., Springer-Verlag, Berlin, 2014, pp. 71-83. · Zbl 1382.65448
[38] A. Klawonn, M. Lanser, and O. Rheinbach, {\it Nonlinear FETI-DP and BDDC methods}, SIAM J. Sci. Comput., 36 (2014), pp. A737-A765. · Zbl 1296.65178
[39] A. Klawonn, M. Lanser, and O. Rheinbach, {\it FE2TI (ex_nl/\(fe^2)\) EXASTEEL–Bridging Scales for Multiphase Steels}, http://www.fz-juelich.de/ias/jsc/EN/Expertise/High-Q-Club/FE2TI/_node.html (2015).
[40] A. Klawonn, M. Lanser, and O. Rheinbach, {\it A nonlinear FETI-DP method with an inexact coarse problem}, in Domain Decomposition Methods in Science and Engineering XXII, R. Krause, M. J. Gander, Th. Dickopf, L. F. Pavarino, and L. Halpern, eds., Lect. Notes Comput. Sci. Eng., Springer-Verlag, Berlin, to appear. · Zbl 1339.65218
[41] A. Klawonn, M. Lanser, O. Rheinbach, H. Stengel, and G. Wellein, {\it Hybrid MPI/OpenMP parallelization in FETI-DP methods}, in Proceedings of the Conference on Recent Trends in Computational Engineering (CE2014), Lect. Notes Comput. Sci. Eng. 105, Springer-Verlag, Berlin, 2015, pp. 67-84.
[42] A. Klawonn and O. Rheinbach, {\it A parallel implementation of dual-primal FETI methods for three-dimensional linear elasticity using a transformation of basis}, SIAM J. Sci. Comput., 28 (2006), pp. 1886-1906. · Zbl 1124.74049
[43] A. Klawonn and O. Rheinbach, {\it Inexact FETI-DP methods}, Internat. J. Numer. Methods Engrg., 69 (2007), pp. 284-307. · Zbl 1194.74420
[44] A. Klawonn and O. Rheinbach, {\it Robust FETI-DP methods for heterogeneous three dimensional elasticity problems}, Comput. Methods Appl. Mech. Engrg., 196 (2007), pp. 1400-1414. · Zbl 1173.74428
[45] A. Klawonn and O. Rheinbach, {\it A hybrid approach to 3-level FETI}, PAMM Proc. Appl. Math. Mech., 8 (2008), pp. 10841-10843. · Zbl 1392.65114
[46] A. Klawonn and O. Rheinbach, {\it Highly scalable parallel domain decomposition methods with an application to biomechanics}, ZAMM Z. Angew. Math. Mech., 90 (2010), pp. 5-32. · Zbl 1355.65169
[47] A. Klawonn and O. B. Widlund, {\it Dual-primal FETI methods for linear elasticity}, Comm. Pure Appl. Math., 59 (2006), pp. 1523-1572. · Zbl 1110.74053
[48] A. Klawonn, O. B. Widlund, and M. Dryja, {\it Dual-primal FETI methods for three-dimensional elliptic problems with heterogeneous coefficients}, SIAM J. Numer. Anal., 40 (2002), pp. 159-179. · Zbl 1032.65031
[49] J. Li and O. B. Widlund, {\it FETI-DP, BDDC, and block Cholesky methods}, Internat. J. Numer. Methods Engrg., 66 (2006), pp. 250-271.
[50] J. Mandel and C. R. Dohrmann, {\it Convergence of a balancing domain decomposition by constraints and energy minimization}, Numer. Linear Algebra Appl., 10 (2003), pp. 639-659. · Zbl 1071.65558
[51] J. Mandel, C. R. Dohrmann, and R. Tezaur, {\it An algebraic theory for primal and dual substructuring methods by constraints}, Appl. Numer. Math., 54 (2005), pp. 167-193. · Zbl 1076.65100
[52] J. Mandel, B. Sousedík, and C. R. Dohrmann, {\it On multilevel BDDC}, in Domain Decomposition Methods in Science and Engineering XVII, U. Langer, M. Discacciati, D. E. Keyes, O. B. Widlund, and W. Zulehner, eds., Lect. Notes Comput. Sci. Eng. 60, Springer, Berlin, Heidelberg, 2008, pp. 287-294. · Zbl 1142.65457
[53] J. Pebrel, C. Rey, and P. Gosselet, {\it A nonlinear dual-domain decomposition method: Application to structural problems with damage}, Internat. J. Multiscale Comp. Eng., 6 (2008), pp. 251-262.
[54] O. Rheinbach, {\it Parallel iterative substructuring in structural mechanics}, Arch. Comput. Methods Eng., 16 (2009), pp. 425-463. · Zbl 1179.74157
[55] U. Rüde, {\it Terra-neo–Integrated Co-design of an Exa-scale Earth Mantle Modeling Framework}, http://www.fz-juelich.de/ias/jsc/EN/Expertise/High-Q-Club/Terra-Neo/_node.html (2014).
[56] B. F. Smith, P. E. Bjørstad, and W. Gropp, {\it Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations}, Cambridge University Press, Cambridge, UK, 1996. · Zbl 0857.65126
[57] B. Sousedík and J. Mandel, {\it On adaptive-multilevel BDDC}, in Domain Decomposition Methods in Science and Engineering XIX, Lect. Notes Comput. Sci. Eng. 78, Springer, Heidelberg, 2011, pp. 39-50. · Zbl 1217.65231
[58] A. Toselli and O. Widlund, {\it Domain Decomposition Methods–Algorithms and Theory}, Springer Ser. Comput. Math. 34, Springer, Berlin, 2004. · Zbl 1069.65138
[59] U. Trottenberg, C. W. Oosterlee, and A. Schüller, {\it Multigrid}, Academic Press, London, San Diego, 2001.
[60] X. Tu, {\it Three-level BDDC in three dimensions}, SIAM J. Sci. Comput., 29 (2007), pp. 1759-1780. · Zbl 1163.65094
[61] X. Tu, {\it Three-level BDDC in two dimensions}, Inter. J. Numer. Methods Engrg., 69 (2007), pp. 33-59. · Zbl 1134.65087
[62] O. C. Zienkiewicz and R. L. Taylor, {\it The Finite Element Method for Solid and Structural Mechanics}, Elsevier, Oxford, UK, 2005. · Zbl 1084.74001
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.