A scalable framework for the partitioned solution of fluid-structure interaction problems. (English) Zbl 1464.74232

Summary: In this work, we present a scalable and efficient parallel solver for the partitioned solution of fluid-structure interaction problems through multi-code coupling. Two instances of an in-house parallel software, TermoFluids, are used to solve the fluid and the structural sub-problems, coupled together on the interface via the preCICE coupling library. For fluid flow, the Arbitrary Lagrangian-Eulerian form of the Navier-Stokes equations is solved on an unstructured conforming grid using a second-order finite-volume discretization. A parallel dynamic mesh method for unstructured meshes is used to track the moving boundary. For the structural problem, the nonlinear elastodynamics equations are solved on an unstructured grid using a second-order finite-volume method. A semi-implicit FSI coupling method is used which segregates the fluid pressure term and couples it strongly to the structure, while the remaining fluid terms and the geometrical nonlinearities are only loosely coupled. A robust and advanced multi-vector quasi-Newton method is used for the coupling iterations between the solvers. Both the fluid and the structural solver use distributed-memory parallelism. The intra-solver communication required for data update in the solution process is carried out using non-blocking point-to-point communicators. The inter-code communication is fully parallel and point-to-point, avoiding any central communication unit. Inside each single-physics solver, the load is balanced by dividing the computational domain into fairly equal blocks for each process. Additionally, a load balancing model is used at the inter-code level to minimize the overall idle time of the processes. Two practical test cases in the context of hemodynamics are studied, demonstrating the accuracy and computational efficiency of the coupled solver. Strong scalability test results show a parallel efficiency of 83% on \(10,080\) CPU cores.


74S10 Finite volume methods applied to problems in solid mechanics
74F10 Fluid-solid interactions (including aero- and hydro-elasticity, porosity, etc.)
76M12 Finite volume methods applied to problems in fluid mechanics
Full Text: DOI


[1] Bazilevs, Y.; Calo, VM; Zhang, Y.; Hughes, TJR, Isogeometric fluid-structure interaction analysis with applications to arterial blood flow, Comput Mech, 38, 4-5, 310-322 (2006) · Zbl 1161.74020
[2] Takizawa, K.; Bazilevs, Y.; Tezduyar, TE, Space-time and ALE-VMS techniques for patient-specific cardiovascular fluid-structure interaction modeling, Arch Comput Methods Eng, 19, 2, 171-225 (2012) · Zbl 1354.92023
[3] Degroote, J., Partitioned simulation of fluid-structure interaction, Arch Comput Methods Eng, 20, 185-238 (2013) · Zbl 1354.74066
[4] Hou, G.; Wang, J.; Layton, A., Numerical methods for fluid-structure interaction—a review, Commun Comput Phys, 12, 2, 337-377 (2012) · Zbl 1373.76001
[5] Kong, F.; Cai, X-C, A scalable nonlinear fluid-structure interaction solver based on a Schwarz preconditioner with isogeometric unstructured coarse spaces in 3d, J Comput Phys, 340, 498-518 (2017) · Zbl 1376.76022
[6] Deparis, S.; Forti, D.; Grandperrin, G.; Quarteroni, A., FaCSI: a block parallel preconditioner for fluid-structure interaction in hemodynamics, J Comput Phys, 327, 700-718 (2016) · Zbl 1373.74036
[7] Kong, F.; Kheyfets, V.; Finol, E.; Cai, X-C, Simulation of unsteady blood flows in a patient-specific compliant pulmonary artery with a highly parallel monolithically coupled fluid-structure interaction algorithm, Int J Numer Methods Biomed Eng, 35, 7, e3208 (2019)
[8] Kataoka, S.; Minami, S.; Kawai, H.; Yamada, T.; Yoshimura, S., A parallel iterative partitioned coupling analysis system for large-scale acoustic fluid-structure interactions, Comput Mech, 53, 6, 1299-1310 (2014) · Zbl 1311.74040
[9] Cajas, J.; Houzeaux, G.; Vázquez, M.; Garcia, M.; Casoni, E.; Calmet, H.; Artigues, A.; Borrell, R.; Lehmkuhl, O.; Pastrana, D., Fluid-structure interaction based on HPC multicode coupling, SIAM J Sci Comput, 40, 6, C677-C703 (2018) · Zbl 1402.74031
[10] Hewitt, S.; Margetts, L.; Revell, A.; Pankaj, P.; Levrero-Florencio, F., OpenFPCI: a parallel fluid-structure interaction framework, Comput Phys Commun, 244, 469-482 (2019)
[11] Larson, J.; Jacob, R.; Ong, E., The model coupling toolkit: a new Fortran90 toolkit for building multiphysics parallel coupled models, Int J High Perform Comput Appl, 19, 3, 277-292 (2005)
[12] Bungartz, H-J; Lindner, F.; Gatzhammer, B.; Mehl, M.; Scheufele, K.; Shukaev, A.; Uekermann, B., preCICE—a fully parallel library for multi-physics surface coupling, Comput Fluids, 141, 250-258 (2016) · Zbl 1390.76004
[13] Thomas, D.; Cerquaglia, ML; Boman, R.; Economon, TD; Alonso, JJ; Dimitriadis, G.; Terrapon, VE, CUPyDO—an integrated Python environment for coupled fluid-structure simulations, Adv Eng Softw, 128, 69-85 (2019)
[14] Bungartz, H-J; Lindner, F.; Mehl, M.; Scheufele, K.; Shukaev, A.; Uekermann, B.; Bungartz, HJ; Neumann, P.; Nagel, WE, Partitioned fluid-structure-acoustics interaction on distributed data: coupling via precice, Software for exascale computing—SPPEXA 2013-2015, 239-266 (2016), Cham: Springer, Cham
[15] Cerquaglia, ML; Thomas, D.; Boman, R.; Terrapon, V.; Ponthot, J-P, A fully partitioned Lagrangian framework for fsi problems characterized by free surfaces, large solid deformations and displacements, and strong added-mass effects, Comput Methods Appl Mech Eng, 348, 409-442 (2019) · Zbl 1440.74380
[16] Causin, P.; Gerbeau, JF; Nobile, F., Added-mass effect in the design of partitioned algorithms for fluid-structure problems, Comput Methods Appl Mech Eng, 194, 4506-4527 (2005) · Zbl 1101.74027
[17] Förster, C.; Wall, WA; Ramm, E., Artificial added mass instabilities in sequential staggered coupling of nonlinear structures and incompressible viscous flows, Comput Methods Appl Mech Eng, 196, 1278-1293 (2007) · Zbl 1173.74418
[18] Fernández, MA; Gerbeau, J-F; Grandmont, C., A projection semi-implicit scheme for the coupling of an elastic structure with an incompressible fluid, Int J Numer Meth Eng, 69, 4, 794-821 (2007) · Zbl 1194.74393
[19] Naseri, A.; Lehmkuhl, O.; Gonzalez, I.; Bartrons, E.; Pérez-Segarra, CD; Oliva, A., A semi-implicit coupling technique for fluid-structure interaction problems with strong added-mass effect, J Fluids Struct, 80, 94-112 (2018)
[20] Naseri, A.; Gonzalez, I.; Amani, A.; Pérez-Segarra, CD; Oliva, A., A second-order time accurate semi-implicit method for fluid-structure interaction problems, J Fluids Struct, 86, 135-155 (2019)
[21] TermoFluids SL (2020) http://www.termofluids.com
[22] Rodriguez, I.; Borell, R.; Lehmkuhl, O.; Perez Segarra, CD; Oliva, A., Direct numerical simulation of the flow over a sphere at Re = 3700, J Fluid Mech, 679, 263-287 (2011) · Zbl 1241.76302
[23] Rodrguez, I.; Lehmkuhl, O.; Chiva, J.; Borrell, R.; Oliva, A., On the flow past a circular cylinder from critical to super-critical reynolds numbers: wake topology and vortex shedding, Int J Heat Fluid Flow, 55, 91-103 (2015)
[24] Balcázar, N.; Jofre, L.; Lehmkuhl, O.; Castro, J.; Rigola, J., A finite-volume/level-set method for simulating two-phase flows on unstructured grids, Int J Multiph Flow, 64, 55-72 (2014)
[25] Gutiérrez, E.; Favre, F.; Balcazar, N.; Amani, A.; Rigola, J., Numerical approach to study bubbles and drops evolving through complex geometries by using a level set-moving mesh-immersed boundary method, Chem Eng J, 349, 662-682 (2018)
[26] Galione, P.; Lehmkuhl, O.; Rigola, J.; Oliva, A., Fixed-grid numerical modeling of melting and solidification using variable thermo-physical properties—application to the melting of n-octadecane inside a spherical capsule, Int J Heat Mass Transf, 86, 721-743 (2015)
[27] Bartrons, E.; Oliet, C.; Gutierrez, E.; Naseri, A.; Pérez-Segarra, CD, A finite volume method to solve the frost growth using dynamic meshes, Int J Heat Mass Transf, 124, 615-628 (2018)
[28] Colomer, G.; Borrell, R.; Trias, FX; Rodríguez, I., Parallel algorithms for Sn transport sweeps on unstructured meshes, J Comput Phys, 232, 1, 118-135 (2013)
[29] Jofre, L.; Borrell, R.; Lehmkuhl, O.; Oliva, A., Parallel load balancing strategy for volume-of-fluid methods on 3-d unstructured meshes, J Comput Phys, 282, 269-288 (2015) · Zbl 1351.76217
[30] Borrell, R.; Chiva, J.; Lehmkuhl, O.; Oyarzun, G.; Rodríguez, I.; Oliva, A., Optimising the termofluids CFD code for petascale simulations, Int J Comput Fluid Dyn, 30, 6, 425-430 (2016)
[31] Verstappen, R.; Veldman, A., Symmetry-preserving discretization of turbulent flow, J Comput Phys, 187, 1, 343-368 (2003) · Zbl 1062.76542
[32] Trias, FX; Lehmkuhl, O.; Oliva, A.; Pérez-Segarra, CD; Verstappen, R., Symmetry-preserving discretization of Navier-Stokes equations on collocated unstructured grids, J Comput Phys, 258, 246-267 (2014) · Zbl 1349.65386
[33] Jofre, L.; Lehmkuhl, O.; Ventosa, J.; Trias, FX; Oliva, A., Conservation properties of unstructured finite-volume mesh schemes for the Navier-Stokes equations, Numer Heat Transf Part B Fundam, 65, 1, 53-79 (2014)
[34] Thomas, P.; Lombard, C., Geometric conservation law and its application to flow computations on moving grids, AIAA J, 17, 10, 1030-1037 (1979) · Zbl 0436.76025
[35] Lesoinne, M.; Farhat, C., Geometric conservation laws for flow problems with moving boundaries and deformable meshes, and their impact on aeroelastic computations, Comput Methods Appl Mech Eng, 134, 1-2, 71-90 (1996) · Zbl 0896.76044
[36] Estruch, O.; Lehmkuhl, O.; Borrell, R.; Segarra, CDP; Oliva, A., A parallel radial basis function interpolation method for unstructured dynamic meshes, Comput Fluids, 80, 44-54 (2013) · Zbl 1284.76091
[37] Cardiff P, Demirdžić I Thirty years of the finite volume method for solid mechanics. arXiv preprint arXiv:1810.02105
[38] Jasak H (1996) Error analysis and estimation for the finite volume method with applications to fluid flows. PhD thesis, Imperial College London (University of London)
[39] Macleod, AJ, Acceleration of vector sequences by multidimensional \(\varDelta^2\) methods, Commun Appl Numer Methods, 2, 4, 385-392 (1986) · Zbl 0608.65003
[40] Tuković, Ž.; Ivanković, A.; Karač, A., Finite-volume stress analysis in multi-material linear elastic body, Int J Numer Meth Eng, 93, 4, 400-419 (2013) · Zbl 1352.74010
[41] Cardiff, P.; Tuković, H.; Jasak, A.; Ivanković, A., Block-coupled finite volume methodology for linear elasticity and unstructured meshes, Comput Struct, 175, 100-122 (2016)
[42] Chandrashekar, P.; Garg, A., Vertex-centroid finite volume scheme on tetrahedral grids for conservation laws, Comput Math Appl, 65, 1, 58-74 (2013) · Zbl 1268.65121
[43] González I, Naseri A, Chiva J, Rigola J, Pérez-Segarra CD (2018) An enhanced finite volume based solver for thermoelastic materials in fluid-structure coupled problems. In: 6th European conference on computational mechanics (ECCM 6), 7th European conference on computational fluid dynamics (ECFD 7), Glasgow, UK, vol 15, pp 1115-1127
[44] Küttler, U.; Wall, WA, Fixed-point fluid-structure interaction solvers with dynamic relaxation, Comput Mech, 43, 61-72 (2008) · Zbl 1236.74284
[45] Gerbeau, JF; Vidrascu, M., A quasi-Newton algorithm based on a reduced model for fluid-structure interaction problems in blood flows, ESAIM Math Model Numer Anal, 37, 631-647 (2003) · Zbl 1070.74047
[46] Michler, C.; Brummelen, EHV; Borst, RD, An interface Newton-Krylov solver for fluid-structure interaction, Int J Numer Meth Fluids, 47, 10-11, 1189-1195 (2005) · Zbl 1069.76033
[47] Scheufele, K.; Mehl, M., Robust multisecant quasi-Newton variants for parallel fluid-structure simulations-and other multiphysics applications, SIAM J Sci Comput, 39, 5, S404-S433 (2017) · Zbl 06799795
[48] Bungartz, H-J; Lindner, F.; Mehl, M.; Uekermann, B., A plug-and-play coupling approach for parallel multi-field simulations, Comput Mech, 55, 6, 1119-1129 (2015) · Zbl 1325.74044
[49] Mehl, M.; Uekermann, B.; Bijl, H.; Blom, D.; Gatzhammer, B.; Van Zuijlen, A., Parallel coupling numerics for partitioned fluid-structure interaction simulations, Comput Math Appl, 71, 4, 869-891 (2016)
[50] Oyarzun, G.; Borrell, R.; Gorobets, A.; Oliva, A., MPI-CUDA sparse matrix-vector multiplication for the conjugate gradient method with an approximate inverse preconditioner, Comput Fluids, 92, 244-252 (2014) · Zbl 1390.65038
[51] Oyarzun, G.; Borrell, R.; Gorobets, A.; Oliva, A., Portable implementation model for CFD simulations. Application to hybrid CPU/GPU supercomputers, Int J Comput Fluid Dyn, 31, 9, 396-411 (2017)
[52] Oyarzun, G.; Borrell, R.; Gorobets, A.; Mantovani, F.; Oliva, A., Efficient CFD code implementation for the ARM-based Mont-Blanc architecture, Future Gener Comput Syst, 79, 786-796 (2018)
[53] Karypis, G.; Kumar, V., A fast and high quality multilevel scheme for partitioning irregular graphs, SIAM J Sci Comput, 20, 1, 359-392 (1998) · Zbl 0915.68129
[54] The HDF Group, Hierarchical Data Format, version 5, http://www.hdfgroup.org/HDF5/ (1997-2019)
[55] Totounferoush A, Ebrahimi Pour N, Schroder J, Roller S, Mehl M (2019) A new load balancing approach for coupled multi-physics simulations. In: IEEE international parallel and distributed processing symposium workshops (IPDPSW). IEEE 2019, pp 676-682. 10.1109/IPDPSW.2019.00115
[56] Calotoiu A, Beckinsale D, Earl CW, Hoefler T, Karlin I, Schulz M, Wolf F (2016) Fast multi-parameter performance modeling. In: 2016 IEEE international conference on cluster computing (CLUSTER). IEEE, pp. 172-181. 10.1109/CLUSTER.2016.57
[57] Leibniz Supercomputing Centre of the Bavarian Academy of Sciences and Humanities , https://www.lrz.de (2019)
[58] Formaggia, L.; Gerbeau, JF; Nobile, F.; Quarteroni, A., On the coupling of 3D and 1D Navier-Stokes equations for flow problems in compliant vessels, Comput Methods Appl Mech Eng, 191, 6-7, 561-582 (2001) · Zbl 1007.74035
[59] Fernández, MA; Moubachir, M., A Newton method using exact Jacobians for solving fluid-structure coupling, Comput Struct, 83, 127-142 (2005)
[60] Eken, A.; Sahin, M., A parallel monolithic algorithm for the numerical simulation of large-scale fluid structure interaction problems, Int J Numer Meth Fluids, 80, 12, 687-714 (2016)
[61] Lozovskiy, A.; Olshanskii, MA; Vassilevski, YV, Analysis and assessment of a monolithic FSI finite element method, Comput Fluids, 179, 277-288 (2019) · Zbl 1411.76068
[62] 2nd CFD challenge predicting patient-specific hemodynamics at rest and stress through an aortic coarctation, http://www.vascularmodel.org/miccai2013/ (2013)
[63] Fernández, MA; Landajuela, M.; Vidrascu, M., Fully decoupled time-marching schemes for incompressible fluid/thin-walled structure interaction, J Comput Phys, 297, 156-181 (2015) · Zbl 1349.76201
[64] Westerhof, N.; Lankhaar, J-W; Westerhof, BE, The arterial Windkessel, Med Biol Eng Comput, 47, 2, 131-141 (2009)
[65] Pant, S.; Fabrèges, B.; Gerbeau, J-F; Vignon-Clementel, I., A methodological paradigm for patient-specific multi-scale CFD simulations: from clinical measurements to parameter estimates for individual analysis, Int J Numer Methods Biomed Eng, 30, 12, 1614-1648 (2014)
[66] Tuković, Ž.; Karač, A.; Cardiff, P.; Jasak, H.; Ivanković, A., OpenFOAM finite volume solver for fluid-solid interaction, Trans FAMENA, 42, 3, 1-31 (2018)
[67] Seo, J.; Schiavazzi, DE; Marsden, AL, Performance of preconditioned iterative linear solvers for cardiovascular simulations in rigid and deformable vessels, Comput Mech, 64, 3, 717-739 (2019) · Zbl 1465.74166
[68] Jodlbauer, D.; Langer, U.; Wick, T., Parallel block-preconditioned monolithic solvers for fluid-structure interaction problems, Int J Numer Meth Eng, 117, 6, 623-643 (2019)
[69] Forti, D.; Quarteroni, A.; Deparis, S., A parallel algorithm for the solution of large-scale nonconforming fluid-structure interaction problems in hemodynamics, J Comput Math, 35, 3, 363-380 (2017) · Zbl 1399.65237
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.