×

Alya: computational solid mechanics for supercomputers. (English) Zbl 1348.74007

Summary: While solid mechanics codes are now conventional tools both in industry and research, the increasingly more exigent requirements of both sectors are fuelling the need for more computational power and more advanced algorithms. For obvious reasons, commercial codes are lagging behind academic codes often dedicated either to the implementation of one new technique, or the upscaling of current conventional codes to tackle massively large scale computational problems. Only in a few cases, both approaches have been followed simultaneously. In this article, a solid mechanics simulation strategy for parallel supercomputers based on a hybrid approach is presented. Hybrid parallelization exploits the thread-level parallelism of multicore architectures, combining MPI tasks with OpenMP threads. This paper describes the proposed strategy, programmed in Alya, a parallel multi-physics code. Hybrid parallelization is specially well suited for the current trend of supercomputers, namely large clusters of multicores. The strategy is assessed through transient non-linear solid mechanics problems, both for explicit and implicit schemes, running on thousands of cores. In order to demonstrate the flexibility of the proposed strategy under advance algorithmic evolution of computational mechanics, a non-local parallel overset meshes method (Chimera-like) is implemented and the conservation of the scalability is demonstrated.

MSC:

74-04 Software, source code, etc. for problems pertaining to mechanics of deformable solids
74Sxx Numerical and other methods in solid mechanics
65N22 Numerical solution of discretized equations for boundary value problems involving PDEs
65Y05 Parallel numerical computation
65Y15 Packaged methods for numerical algorithms
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[1] Alya system. http://www.bsc.es/computer-applications/alya-system · Zbl 1241.92016
[2] Bigdft. http://bigdft.org/Wiki/index.php?title=Presenting_BigDFT
[3] Codeaster. http://www.code-aster.org/ · Zbl 1059.76037
[4] CODE_SATURN. http://code-saturne.org · Zbl 1225.74105
[5] Febio. http://febio.org/
[6] Metis, family of multilevel partitioning algorithms. http://glaros.dtc.umn.edu/gkhome/views/metis · Zbl 1228.74093
[7] Openacc. https://developer.nvidia.com/openacc
[8] Opencl. https://developer.nvidia.com/opencl
[9] Openfoam. http://www.openfoam.com/
[10] Openmp. http://openmp.org/ · Zbl 0986.68722
[11] Openmp. http://openmp.org/wp/ · Zbl 0986.68722
[12] Summary of available software for sparse direct methods. http://www.cise.ufl.edu/research/sparse/codes/
[13] Adams M (1999) Parallel multigrid algorithms for unstructured 3d large deformation elasticity and plasticity finite element problems. Technical report UCB/CSD-99-1036, EECS Department, University of California, Berkeley. http://www.eecs.berkeley.edu/Pubs/TechRpts/1999/5398.html
[14] Adams MF (2007) Algebraic multigrid methods for direct frequency response analyses in solid mechanics. Comput Mech 39:497-507 · Zbl 1163.74043 · doi:10.1007/s00466-006-0047-8
[15] Malan AG, Oxtoby O (2013) An accelerated, fully-coupled, parallel 3d hybrid finite-volume fluid-structure interaction scheme. Comput Methods Appl Mech Eng 253:426-438 · Zbl 1297.74041 · doi:10.1016/j.cma.2012.09.004
[16] Amestoy P, Duff I, L’Excellent JY (2000) Multifrontal parallel distributed symmetric and unsymmetric solvers. Comput Methods Appl Mech Eng 184(2-4):501-520 · Zbl 0956.65017 · doi:10.1016/S0045-7825(99)00242-X
[17] Arbenz P, van Lenthe G, Mennel U, Müller R, Sala M (2008) A scalable multi-level preconditioner for matrix-free \[\mu\] μ-finite element analysis of human bone structures. Int J Numer Methods Eng 73:937-947. doi: 10.1002/nme.2101 · Zbl 1262.74031 · doi:10.1002/nme.2101
[18] Badia S, Martin A, Principe J (2014) A highly scalable parallel implementation of balancing domain decomposition by constraints. SIAM J Sci Comput 36(2):C190-C218 · Zbl 1296.65177 · doi:10.1137/130931989
[19] Balay S, Abhyankar S, Adams MF, Brown J, Brune P, Buschelman K, Eijkhout V, Gropp WD, Kaushik D, Knepley MG, McInnes LC, Rupp K, Smith BF, Zhang H (2014) PETSc Web page. http://www.mcs.anl.gov/petsc · Zbl 1036.65045
[20] Balay S, Adams MF, Brown J, Brune P, Buschelman K, Eijkhout V, Gropp WD, Kaushik D, Knepley MG, McInnes LC, Rupp K, Smith BF, Zhang H (2013) PETSc users manual. Technical report ANL-95/11—revision 3.4, Argonne National Laboratory. http://www.mcs.anl.gov/petsc
[21] Becker G, Noels L (2013) A full-discontinuous galerkin formulation of nonlinear kirchhoff-love shells: elasto-plastic finite deformations, parallel computation, and fracture applications. Int J Numer Methods Eng 93:80-117. doi:10.1002/nme.4381 · Zbl 1352.74170 · doi:10.1002/nme.4381
[22] Benek J (1986) Chimera. A grid-embedding technique. Technical report, DTIC Document
[23] Bhardwaj M, Pierson K, Reese G, Walsh T, Day D, Alvin K, Peery J, Farhat C, Lesoinne M (2002) Salinas: a scalable software for high-performance structural and solid mechanics simulations. In: Proceedings of the 2002 ACM/IEEE conference on supercomputing., SC’02IEEE Computer Society Press, Los Alamitos, pp 1-19 · Zbl 0606.73096
[24] Blatt M (2009) Dune on bluegeen/p. In: Proceedings of 15th SciComp. University Press · Zbl 1261.76030
[25] Casasdei F, Avotins J (1997) A language for implementing computational mechanics applications. In: Technology of object-oriented languages and systems, 1997. TOOLS 25, Proceedings, pp 52-67 · Zbl 1284.76250
[26] Ciccozzi F (2013) Towards code generation from design models for embedded systems on heterogeneous cpu-gpu platforms. In: 2013 IEEE 18th conference on emerging technologies factory automation (ETFA), pp 1-4 · Zbl 0738.65014
[27] Cirak F, Deiterding R, Mauch S (2007) Large-scale fluidstructure interaction simulation of viscoplastic and fracturing thin-shells subjected to shocks and detonations. Comput Struct 85:1049-1065. doi:10.1016/j.compstruc.2006.11.014 · doi:10.1016/j.compstruc.2006.11.014
[28] Duff I, Reid J (1983) The multifrontal solution of indefinite sparse symmetric linear-equations. ACM Trans Math Softw 9(3):302-325 · Zbl 0515.65022 · doi:10.1145/356044.356047
[29] Eguzkitza B, Houzeaux G, Aubry R, Owen H, Vázquez M (2013) A parallel coupling strategy for the Chimera and domain decomposition methods in computational mechanics. Comput Fluids 80:128-141 · Zbl 1284.65166 · doi:10.1016/j.compfluid.2012.04.018
[30] El maliki A, Fortin M, Tardieu N, Fortin A (2010) Iterative solvers for 3d linear and nonlinear elasticity problems: Displacement and mixed formulations. Int J Numer Methods Eng 83(13):1780- 1802 · Zbl 1202.74005
[31] Falgout RD, Yang UM (2002) hypre: a library of high performance preconditioners. In: Preconditioners, lecture notes in computer science, pp 632-641 · Zbl 1056.65046
[32] Farhat C, Roux FX, Oden JT (1994) Implicit parallel processing in structural mechanics. Elsevier Science SA, Amsterdam
[33] Flaig C, Arbenz P (2011) A scalable memory efficient multigrid solver for micro-finite element analyses based on CT images. Parallel Comput 37:846-854. doi:10.1016/j.parco.2011.08.001 · doi:10.1016/j.parco.2011.08.001
[34] George A, Liu JW (1981) Computer solution of large sparse positive definite. Prentice Hall, Englewood cliffs (Professional technical reference) · Zbl 0516.65010
[35] Gerstenberger A, Tuminaro R (2013) An algebraic multigrid approach to solve extended finite element method based fracture problems. Int J Numer Methods Eng 94:248-272. doi:10.1002/nme.4442 · Zbl 1352.74355 · doi:10.1002/nme.4442
[36] Geuzaine C, Remacle JF (2009) Gmsh: a 3-d finite element mesh generator with built-in pre-and post-processing facilities. Int J Numer Methods Eng 79(11):1309-1331 · Zbl 1176.74181 · doi:10.1002/nme.2579
[37] Goudreau G, Hallquist J (1982) Recent developments in large-scale finite element Lagrangian hydrocode technology. Comput Methods Appl Mech Eng 33:725-757. doi:10.1016/0045-7825(82)90129-3 · Zbl 0493.73072 · doi:10.1016/0045-7825(82)90129-3
[38] Gould NIM, Scott JA, Hu Y (2007) A numerical evaluation of sparse direct solvers for the solution of large sparse symmetric linear systems of equations. ACM Trans Math Softw 33(2):300-331
[39] Grinberg L, Pekurovsky D, Sherwin SJ, Karniadakis GE (2009) Parallel performance of the coarse space linear vertex solver and low energy basis preconditioner for spectral/hp elements. Parallel Comput 35(5):284-304 · doi:10.1016/j.parco.2008.12.002
[40] Gupta A, Koric S, George T (2009) Sparse matrix factorization on massively parallel computers. In: Proceedings of the conference on high performance computing networking, storage and analysis, SC’09, ACM, New York, pp 1:1-1:12 · Zbl 0493.73072
[41] Goddeke D, Wobker H, Strzodka R, Mohd-Yusof J, McCormick P, Turek S (2009) Co-processor acceleration of an unmodified parallel solid mechanics code with feastgpu. Int J Comput Sci Eng 4(4):254-269 · Zbl 1284.65166
[42] Hales JD, Novascone SR, Williamson RL, Gaston DR, Tonks MR (2012) Solving nonlinear solid mechanics problems with the Jacobian-free Newton Krylov method. Comput Model Eng Sci 84:84-123 · Zbl 1356.74202
[43] Heath M, Ng E, Peyton B (1991) Parallel algorithms for sparse linear systems. SIAM Rev 33(3):420-460 · Zbl 0738.65014 · doi:10.1137/1033099
[44] Heil M, Hazel A, Boyle J (2008) Solvers for large-displacement fluid-structure interaction problems: segregated versus monolithic approaches. Comput Mech 43(1):91-101 · Zbl 1309.76126 · doi:10.1007/s00466-008-0270-6
[45] Heroux MA, Bartlett RA, Howle VE, Hoekstra RJ, Hu JJ, Kolda TG, Lehoucq RB, Long KR, Pawlowski RP, Phipps ET, Salinger AG, Thornquist HK, Tuminaro RS, Willenbring JM, Williams A, Stanley KS (2005) An overview of the Trilinos project. ACM Trans Math Softw 31(3):397-423 · Zbl 1136.65354 · doi:10.1145/1089014.1089021
[46] Hess B, Kutzner C, van der Spoel D, Lindahl E (2008) GROMACS 4: algorithms for highly efficient, load-balanced, and scalable molecular simulation. J Chem Theory Comput 4(3), 435-447 (2008) doi:10.1021/ct700301q · Zbl 1163.74043
[47] Houzeaux G, de la Cruz R, Owen H, Vázquez M (2013) Parallel uniform mesh multiplication applied to a Navier-Stokes solver. Comput Fluids 80:142-151 · Zbl 1284.76250 · doi:10.1016/j.compfluid.2012.04.017
[48] Houzeaux G, Vázquez M, Aubry R, Cela J (2009) A massively parallel fractional step solver for incompressible flows. JCP 228(17):6316-6332 · Zbl 1261.76030 · doi:10.1016/j.jcp.2009.05.019
[49] Hughes T, Ferencz R (1987) Large-scale vectorized implicit calculations in solid mechanics on a CrayX-MP/48 utilizing EBE preconditioned conjugate gradients. Comput Methods Appl Mech Eng 61:215-248. doi:10.1016/0045-7825(87)90005-3 · Zbl 0606.73096 · doi:10.1016/0045-7825(87)90005-3
[50] Hughes TJ (2012) The finite element method: linear static and dynamic finite element analysis. DoverPublications.com · Zbl 0607.76061
[51] Hussain M, Abid M, Ahmad M, Khokhar A, Masud A (2011) A parallel implementation of ALE moving mesh technique for FSI problems using OpenMP. Int J Parallel Program 39:717-745. doi:10.1007/s10766-011-0168-3 · doi:10.1007/s10766-011-0168-3
[52] Jouglard C, Coutinho A (1998) A comparison of iterative multi-level finite element solvers. Comput Struct 69(5):655-670 · Zbl 0941.74064 · doi:10.1016/S0045-7949(98)00123-0
[53] Kilic SA, Saied F, Sameh A (2004) Efficient iterative solvers for structural dynamics problems. Comput Struct 82(28):2363-2375 · doi:10.1016/j.compstruc.2004.06.001
[54] Knoll D, Keyes D (2004) Jacobian-free Newton Krylov methods: a survey of approaches and applications. J Comput Phys 193(2):357-397 · Zbl 1036.65045 · doi:10.1016/j.jcp.2003.08.010
[55] Komatitsch D, Erlebacher G, Göddeke D, Michéa D (2010) High-order finite-element seismic wave propagation modeling with MPI on a large GPU cluster. J Comput Phys 229:7692-7714. doi:10.1016/j.jcp.2010.06.024 · Zbl 1194.86019 · doi:10.1016/j.jcp.2010.06.024
[56] Lafortune P, Arís R, Vázquez M, Houzeaux G (2012) Coupled electromechanical model of the heart: parallel finite element formulation. Int J Numer Methods Biomed Eng 28:72-86. doi:10.1002/cnm.1494 · Zbl 1242.92015 · doi:10.1002/cnm.1494
[57] Lang S, Wieners C, Wittum G (2002) The application of adaptive parallel multigrid methods to problems in nonlinear solid mechanics. In: Ramm E, Rank E, Rannacher R, Schweizerhof K, Stein E, Wendland W, Wittum G, Wriggers P, Wunderlich W (eds) Error-controlled adaptive finite elements in solid mechanics, 422 pp, ISBN: 978-0-471-49650-2
[58] Li X, Demmel JW (2003) Superlu dist: a scalable distributed-memory sparse direct solver for unsymmetric linear systems. ACM Trans Math Softw 29:110-140 · Zbl 1068.90591 · doi:10.1145/779359.779361
[59] Liu WK, Belytschko T, Moran B (2000) Nonlinear finite elements for continua and structures. Wiley, New York · Zbl 0959.74001
[60] Liu Y, Zhou W, Yang Q (2007) A distributed memory parallel element-by-element scheme based on Jacobi-conditioned conjugate gradient for 3D finite element analysis. Finite Elem Anal Design 43:494-503. doi:10.1016/j.finel.2006.12.007 · doi:10.1016/j.finel.2006.12.007
[61] Logg A, Mardal KA, Wells GN (eds) (2012) Automated solution of differential equations by the finite element method. Lecture notes in computational science and engineering, vol 84. Springer, Berlin. doi:10.1007/978-3-642-23099-8 · Zbl 1247.65105
[62] Lohner R, Mut F, Cebral J, Aubry R, Houzeaux G (2010) Deflated preconditioned conjugate gradient solvers for the pressure-poisson equation: Extensions and improvements. Int J Numer Meth Eng 10196-10208 · Zbl 1153.76035
[63] Luebke D (2008) Cuda: scalable parallel programming for high-performance scientific computing. In: 5th IEEE international symposium on biomedical imaging: from nano to macro, 2008. ISBI 2008, pp 836-838 · Zbl 1241.92016
[64] Maday Y, Magoulès F (2006) Absorbing interface conditions for domain decomposition methods: a general presentation. Comput Methods Appl Mech Eng 195(29-32):3880-3900 · Zbl 1168.65423 · doi:10.1016/j.cma.2005.01.025
[65] Maurer D, Wieners C (2011) A parallel block LU decomposition method for distributed finite element matrices. Parallel Comput 37(12):742-758 · doi:10.1016/j.parco.2011.05.007
[66] McCormick SF (1987) Multigrid methods. Frontiers in applied mathematics. Philadelphia, Pa. Society for Industrial and Applied Mathematics · Zbl 0659.65094
[67] Moore D, Jérusalem A, Nyein M, Noels L, Jaffee M, Radovitzky R (2009) Computational biology—modeling of primary blast effects on the central nervous system. NeuroImage 47(2):T10-T20. doi:10.1016/j.neuroimage.2009.02.019 · doi:10.1016/j.neuroimage.2009.02.019
[68] Owens J, Houston M, Luebke D, Green S, Stone J, Phillips J (2008) Gpu computing. Proc IEEE 96(5):879-899 · doi:10.1109/JPROC.2008.917757
[69] Paszyski M, Jurczyk T, Pardo D (2013) Multi-frontal solver for simulations of linear elasticity coupled with acoustics. Comput Sci 12(0). http://journals.agh.edu.pl/csci/article/view/102
[70] Quey R, Dawson PR, Barbe F (2011) Large-scale 3D random polycrystals for the finite element method: Generation, meshing and remeshing. Comput Methods Appl Mech Eng 200:1729-1745. doi:10.1016/j.cma.2011.01.002 · Zbl 1228.74093 · doi:10.1016/j.cma.2011.01.002
[71] Radovitzky R, Seagraves A, Tupek M, Noels L (2011) A scalable 3D fracture and fragmentation algorithm based on a hybrid, discontinuous galerkin, cohesive element method. Comput Methods Appl Mech Eng 200:326-344. doi:10.1016/j.cma.2010.08.014 · Zbl 1225.74105 · doi:10.1016/j.cma.2010.08.014
[72] Saad Y (2003) Iterative methods for sparse linear systems. Society for Industrial and Applied Mathematics · Zbl 1031.65046
[73] Smith BF (1995) Domain decomposition methods for partial differential equations. In: Proceedings of ICASE/LaRC Workshop on Parallel Numerical Algorithms. University Press
[74] Soto O, Löhner R, Camelli F (2003) A linelet preconditioner for incompressible flow solvers. Int J Numer Meth Heat Fluid Flow 13(1):133-147 · Zbl 1059.76037 · doi:10.1108/09615530310456796
[75] Steger J, Benek FDJ (1983) A chimera grid scheme. Adv Grid Gener 5:59-69
[76] Steger J, Benek J (1987) On the use of composite grid schemes in computational aerodynamics. Comput Meth Appl Mech Eng 64:301-320 · Zbl 0607.76061 · doi:10.1016/0045-7825(87)90045-4
[77] Stewart JR, Edwards H (2004) A framework approach for developing parallel adaptive multiphysics applications. Finite Elem Anal Design 40(12):1599-1617 (The Fifteenth Annual Robert J. Melosh Competition)
[78] Kalro V, Tezduyar TE (2000) A parallel 3d computational method for fluid-structure interactions in parachute systems. Comput Methods Appl Mech Eng 190:1467-1482 · Zbl 0993.76044 · doi:10.1016/S0045-7825(00)00204-8
[79] van Rietbergen B, Weinans H, Huiskes R, Polman B (1996) Computational strategies for iterative solutions of large FEM applications employing voxel data. Int J Numer Methods Eng 39:2743-2767. doi:10.1002/(SICI)1097-0207(19960830)39:162743:AID-NME9743.3.CO;2-1 · Zbl 0883.73079
[80] Vázquez M, Arís R, Houzeaux G, Aubry R, Villar P, Garcia-Barnós J, Gil D, Carreras F (2011) A massively parallel computational electrophysiology model of the heart. Int J Numer Methods Biomed Eng 27(12):1911-1929. doi:10.1002/cnm.1443 · Zbl 1241.92016 · doi:10.1002/cnm.1443
[81] Vázquez M, Houzeaux G, Grima R, Cela J (2007) Applications of parallel computational fluid mechanics in MareNostrum supercomputer: low-mach compressible flows. In: PARCFD2007. Antalya (Turkey)
[82] Vázquez M, Rubio F, Houzeaux G, González J, Giménez J, Beltran V, de la Cruz R, Folch A (2014) Xeon phi performance for hpc-based computational mechanics codes
[83] Waisman H, Berger-Vergiat L (2013) An adaptive domain decomposition preconditioner for crack propagation problems modeled by XFEM. Int J Multiscale Comput Eng 11(6):633-654 · doi:10.1615/IntJMultCompEng.2013006012
[84] Wall WA, Ramm E. Fluid-structure interaction based upon stabilized (ale) finite element method. In: IV World congress on computational mechanics. Barcelona. CIMNE.
[85] Wieners C (2001) The application of multigrid methods to plasticity at finite strains. ZAMM J Appl Math Mech [Zeitschrift fr Angewandte Mathematik und Mechanik] 81(S3):733-734 · Zbl 0998.74016 · doi:10.1002/zamm.200108115142
[86] Zienkiewicz O, Taylor R (2000) The Finite Elements method for Solid and Structural Mechanics, 5th edn. Butterworth-Heinermann, Boston · Zbl 0991.74003
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.