×

zbMATH — the first resource for mathematics

HPC\(^2\) – a fully-portable, algebra-based framework for heterogeneous computing. Application to CFD. (English) Zbl 1410.76323
Summary: The variety of computing architectures competing in the exascale race makes the portability of codes of major importance. In this work, the HPC\(^{2}\) code is presented as a fully-portable, algebra-based framework suitable for heterogeneous computing. In its application to CFD, the algorithm of the time-integration phase relies on a reduced set of only three algebraic operations: the sparse matrix-vector product, the linear combination of vectors and the dot product. This algebraic approach combined with a multilevel MPI+OpenMP+OpenCL parallelization naturally provides portability. The performance has been studied on different architectures including multicore CPUs, Intel Xeon Phi accelerators and GPUs of AMD and NVIDIA. The multi-GPU scalability is demonstrated up to 256 devices. The heterogeneous execution is tested on a CPU+GPU hybrid cluster. Finally, results of the direct numerical simulation of a turbulent flow in a 3D air-filled differentially heated cavity are presented to show the capabilities of the HPC\(^{2}\) dealing with large-scale CFD simulations.

MSC:
76M25 Other numerical methods (fluid mechanics) (MSC2010)
65Y10 Numerical algorithms for specific classes of architectures
Software:
HPCG; OpenCL; HPC2; MPI; PyFR; HOSTA
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Dongarra, J., The international exascale software project roadmap, Int J High Perform Comput Appl, 25, 1, 3-60, (2011)
[2] Rossi, R.; Mossaiby, F.; Idelsohn, S. R., A portable opencl-based unstructured edge-based finite element Navier-Stokes solver on graphics hardware, Comput Fluids, 81, 134-144, (2013) · Zbl 1285.76007
[3] Jacobsen, D. A.; Senocak, I., Multi-level parallelism for incompressible flow computations on GPU clusters, Parallel Comput, 39, 1, 1-20, (2013)
[4] Khajeh-Saeed, A.; Perot, J. B., Direct numerical simulation of turbulence using GPU accelerated supercomputers, J Comput Phys, 235, 241-257, (2013)
[5] Zaspel, P.; Griebel, M., Solving incompressible two-phase flows on multi-GPU clusters, Comput Fluids, 80, 1, 356-364, (2013) · Zbl 1284.76012
[6] Vincent, P.; Witherden, F. D.; Vermeire, B.; Park, J. S.; Iyer, A., Towards Green aviation with python at petascale, International conference for high performance computing, networking, storage and analysis, SC, 1-11, (November 2017)
[7] Dongarra, J.; Heroux, M., HPCG benchmark: a new metric for ranking high performance computing systems, Technical Report June, (2013)
[8] Oyarzun, G.; Borrell, R.; Gorobets, A.; Oliva, A., Portable implementation model for CFD simulations. application to hybrid CPU/GPU supercomputers, Int. J. Comput. Fluid Dyn., 31, 9, 396-411, (2017)
[9] Gorobets, A.; Trias, F. X.; Oliva, A., A parallel MPI+openmp+opencl algorithm for hybrid supercomputations of incompressible flows, Comput. Fluids, 88, 764-772, (2013) · Zbl 1391.76090
[10] Witherden, F. D.; Vermeire, B.; Vincent, P., Heterogeneous computing on mixed unstructured grids with pyfr, Comput Fluids, 120, 173-186, (2015) · Zbl 1390.76014
[11] Xu, C.; Deng, X.; Zhang, L.; Fang, J.; Wang, G.; Jiang, Y.; Cao, W.; Che, Y.; Wang, Y.; Wang, Z.; Liu, W.; Cheng, X., Collaborating CPU and GPU for large-scale high-order CFD simulations with complex grids on the tianhe-1a supercomputer, J Comput Phys, 278, 1, 275-297, (2014) · Zbl 1349.76655
[12] LaSalle, D.; Karypis, G., Multi-threaded graph partitioning, Proceedings - IEEE 27th international parallel and distributed processing symposium, IPDPS 2013, 225-236, (2013)
[13] Cuthill, E.; McKee, J., Reducing the bandwidth of sparse symmetric matrices, ACM ‘69 Proceedings of the 1969 24th national conference, 157-172, (1969)
[14] Oyarzun, G.; Borrell, R.; Gorobets, A.; Mantovani, F.; Oliva, A., Efficient CFD code implementation for the ARM-based mont-blanc architecture, Future Gener Comput Syst, 79, 786-796, (2018)
[15] Saury, D.; Rouger, N.; Djanna, F.; Penot, F., Natural convection in an air-filled cavity: experimental results at large Rayleigh numbers, Int Commun Heat Mass Transfer, 38, 679-687, (2011)
[16] Belleoud, P.; Saury, D.; Lemonnier, D., Coupled velocity and temperature measurements in an air-filled differentially heated cavity at ra=1.2e11, Int J Therm Sci, 123, 151-161, (2018)
[17] Sergent, A.; Joubert, P.; Xin, S.; Le Quéré, P., Resolving the stratification discrepancy of turbulent natural convection in differentially heated air-filled cavities. part II: end walls effects, Int J Heat Fluid Flow, 39, 15-27, (2013)
[18] Salat, J.; Xin, S.; Joubert, P.; Sergent, A.; Penot, F.; Le Quéré, P., Experimental and numerical investigation of turbulent natural convection in a large air-filled cavity, Int J Heat Fluid Flow, 25, 824-832, (2004)
[19] Trias, F. X.; Lehmkuhl, O.; Oliva, A.; Pérez-Segarra, C. D.; Verstappen, R. W.C. P., Symmetry-preserving discretization of Navier-Stokes equations on collocated unstructured meshes, J Comput Phys, 258, 246-267, (2014) · Zbl 1349.65386
[20] Trias, F. X.; Lehmkuhl, O., A self-adaptive strategy for the time-integration of Navier-Stokes equations, Numer Heat Transfer Part B, 60, 2, 116-134, (2011)
[21] Chorin, A. J., Numerical solution of the Navier-Stokes equations, Math Comput, 22, 745-762, (1968) · Zbl 0198.50103
[22] Gorobets, A.; Trias, F. X.; Soria, M.; Oliva, A., A scalable parallel Poisson solver for three-dimensional problems with one periodic direction, Comput Fluids, 39, 525-538, (2010) · Zbl 1242.76203
[23] Trias, F. X.; Gorobets, A.; Soria, M.; Oliva, A., Direct numerical simulation of a differentially heated cavity of aspect ratio 4 with ra-number up to 10^{11} - part I: numerical methods and time-averaged flow, Int J Heat Mass Transfer, 53, 665-673, (2010) · Zbl 1183.80067
[24] Trias, F. X.; Gorobets, A.; Pérez-Segarra, C. D.; Oliva, A., DNS and regularization modeling of a turbulent differentially heated cavity of aspect ratio 5, Int J Heat Mass Transfer, 57, 171-182, (2013)
[25] Barhaghi, D. G.; Davidson, L., Natural convection boundary layer in a 5:1 cavity, Phys Fluids, 19, 12, 125106, (2007) · Zbl 1182.76051
[26] The DNS results presented in this paper are publicly available in http://www.cttc.upc.edu/downloads/DHC_Ra1_2e11.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.