×

MIN3P-HPC: a high-performance unstructured grid code for subsurface flow and reactive transport simulation. (English) Zbl 1465.86002

Summary: The numerical simulation of flow and reactive transport in porous media with complex domains is nontrivial. This paper presents a method to implement fully unstructured grid capabilities into the well-established software ParMIN3P-THCm, a process-based numerical model designed for the investigation of subsurface fluid flow and multicomponent reactive transport in variably saturated porous media with parallelization capability. The enhanced code, MIN3P-HPC, is modularized to support different cell types, spatial discretization methods and gradient reconstruction methods. MIN3P-HPC uses a vertex-centered control volume method with consideration of both vertex-based and cell-based material properties (e.g., permeability). A flexible parallelization scheme based on domain decomposition and thread acceleration was implemented, which allows the use of OpenMP, MPI and hybrid MPI-OpenMP, making optimized use of computer resources ranging from desktop PCs to distributed memory supercomputers. The code was verified by comparing the results obtained with the unstructured grid version to those produced by the structured grid version. Numerical accuracy was also verified against analytical solutions for 2D and 3D solute transport, and by comparison with third-party software using different cell types. Parallel efficiency of OpenMP, MPI and hybrid MPI-OpenMP versions was examined through a series of solute transport and reactive transport test cases. The results demonstrate the versatility and enhanced performance of MIN3P-HPC for reactive transport simulation.

MSC:

86-08 Computational methods for problems pertaining to geophysics
76S05 Flows in porous media; filtration; seepage
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Aavatsmark, I., An introduction to multipoint flux approximations for quadrilateral grids, Comput Geosci, 2, 3-4, 405-432 (2002) · Zbl 1094.76550 · doi:10.1023/A:1021291114475
[2] Appelo, C.; Rolle, M., PHT3D: a reactive multicomponent transportmodel for saturated porous media, Ground Water, 48, 5, 627-632 (2010) · doi:10.1111/j.1745-6584.2010.00732.x
[3] Bain J, Mayer K, Blowes D, Frind E, Molson J, Kahnt R, Jenk U (2001) Modelling the closure-related geochemical evolution of groundwater at a former uranium mine. J Contam Hydrol 52(1):109-135
[4] Balay, S.; Gropp, WD; McInnes, LC; Smith, BF; Arge, E.; Bruaset, AM; Langtangen, HP, Modelling the closure-related geochemical evolution of groundwater at a former uranium mine, Modern software tools in scientific computing, 163-202 (1997), Basel: Birkhäuser Press, Basel · doi:10.1007/978-1-4612-1986-6_8
[5] Balay S, Abhyankar S, Adams MF, Brown J, Brune P, Buschelman K, Dalcin L, Eijkhout V, Gropp WD, Kaushik D, Knepley MG, May DA, McInnes LC, Mills RT, Munson T, Rupp K, Sanan P, Smith BF, Zampini S, Zhang H (2018a) PETSc Web page
[6] Balay S, Abhyankar S, Adams MF, Brown J, Brune P, Buschelman K, Dalcin L, Eijkhout V, Gropp WD, Kaushik D, Knepley MG, May DA, McInnes LC, Mills RT, Munson T, Rupp K, Sanan P, Smith BF, Zampini S, Zhang H (2018b) PETSc users manual. Technical Report ANL-95/11 - Revision 3.9, Argonne National Laboratory
[7] Bas P (1999) Numerical computation of multiphase flows in porous media. Ph.D. thesis, Christian-Albrechts-Universität zu Kiel
[8] Bastian, P.; Helmig, R., Efficient fully-coupled solution techniques for two-phase flow in porous media: parallel multigrid solution and large scale computations, Adv Water Resour, 23, 3, 199-216 (1999) · doi:10.1016/S0309-1708(99)00014-7
[9] Bea, S.; Wilson, S.; Mayer, K.; Dipple, G.; Power, I.; Gamazo, P., Reactive transport modeling of natural carbon sequestration in ultramafic mine tailings, Vadose Zone J, 11, 2, 1-17 (2012) · doi:10.2136/vzj2011.0053
[10] Bea, SA; Mayer, UK; MacQuarrie, KTB, Reactive transport and thermo-hydro-mechanical coupling in deep sedimentary basins affected by glaciation cycles: model development, verification, and illustrative example, Geofluids, 16, 279-300 (2016) · doi:10.1111/gfl.12148
[11] Bea, SA; Su, D.; Mayer, KU; MacQuarriefe, KTB, Evaluation of the potential for dissolved oxygen ingress into deep sedimentary basins during a glaciation event, Geofluids, 2018, 1-20 (2018) · doi:10.1155/2018/9475741
[12] Boufadel, M.; Suidan, M.; Vensona, A., Numerical modeling of water flow below dry salt lakes: effect of capillarity and viscosity, J Hydrol, 221, 1-2, 55-74 (1999) · doi:10.1016/S0022-1694(99)00077-3
[13] Carrayrou, J.; Hoffmann, J.; Knabner, P.; Kräutle, S.; de Dieuleveult, C.; Erhel, J.; Van der Lee, J.; Lagneau, V.; Mayer, K.; MacQuarrie, K., Comparison of numerical methods for simulating strongly nonlinear and heterogeneous reactive transport problems-the momas benchmark case, Comput Geosci, 14, 3, 483-502 (2010) · Zbl 1426.76723 · doi:10.1007/s10596-010-9178-2
[14] Clement TP, Johnson CD (2012) RT3D: reactive transport in 3-dimensions. In: Chapter groundwater reactive transport models chapter 4.16. Bentham Science Publishers, pp 96-111
[15] Despotovic, M.; Nedic, V.; Despotovic, D.; Cvetanovic, S., Evaluation of empirical models for predicting monthly mean horizontal diffuse solar radiation, Renewab Sustain Energy Rev, 56, 246-260 (2016) · doi:10.1016/j.rser.2015.11.058
[16] Farthing, MW; Ogden, FL, Numerical solution of Richards’ equation: a review of advances and challenges, Soil Sci Soc Am J, 81, 1257-1269 (2017) · doi:10.2136/sssaj2017.02.0058
[17] Forsyth, P., A positivity preserving method for simulation of steam injection for napl site remediation, Adv Water Resour, 16, 6, 351-370 (1993) · doi:10.1016/0309-1708(93)90014-7
[18] Freedman, VL; Chen, X.; Finsterle, S.; Freshley, MD; Gorton, I.; Gosink, LJ; Keating, EH; Lansing, CS; Moeglein, WA; Murray, CJ; Pau, GS; Porter, E.; Purohit, S.; Rockhold, M.; Schuchardt, KL; Sivaramakrishnan, C.; Vessilinov, VV; Waichler, SR, A high-performance workflow system for subsurface simulation, Environ Model Softw, 55, 176-189 (2014) · doi:10.1016/j.envsoft.2014.01.030
[19] Fujii A, Nishida A, Oyanagi Y (2005) Evaluation of parallel aggregate creation orders: smoothed aggregation algebraic multigrid method. In: Ng M, BODoncescu A, Yang L, Leng T (eds) High performance computational science and engineering. Volume 172 of IFIP—the international federation for information processing, pp 99-122
[20] Hammond G, Lichtner P, Lu C, RT M (2012) Pflotran: reactive flow and transport code for use on laptops to leadership-class supercomputers. In: Groundwater reactive transport models
[21] Hao Y, Sun Y, Nitao JJ (2011) Overview of nuft: a versatile numerical model for simulating flow and reactive transport in porous media. Groundwater Reactive Transport Models (213-240)
[22] Helmig, R., Multiphase flow and transport processes in the subsurface (1997), Berlin: Springer, Berlin · doi:10.1007/978-3-642-60763-9
[23] Henderson, TH; Mayer, KU; Parker, BL; Al, TA, Three-dimensional density-dependent flow and multicomponent reactive transport modeling of chlorinated solvent oxidation by potassium permanganate, J Contam Hydrol, 106, 3-4, 195-211 (2009) · doi:10.1016/j.jconhyd.2009.02.009
[24] Huyakorn, PS; Thomas, SD; Thompson, BM, Techniques for making finite elements competitve in modeling flow in variably saturated porous media, Water Resour Res, 20, 8, 1099-1115 (1984) · doi:10.1029/WR020i008p01099
[25] Huyakorn, PS; Springer, EP; Guvanasen, V.; Wadsworth, TD, A three-dimensional finite-element model for simulating water flow in variably saturated porous media, Water Resour Res, 22, 13, 1790-1808 (1986) · doi:10.1029/WR022i013p01790
[26] Kang, Q.; Lichtner, PC; Zhang, D., Lattice boltzmann pore-scale model for multicomponent reactive transport in porous media, J Geophys Res, 111, B05203 (2006)
[27] Kolditz, O.; Bauer, S.; Bilke, L.; Böttcher, N.; Delfs, J.; Fischer, T.; Görke, U.; Kalbacher, T.; Kosakowski, G.; McDermott, C.; Park, C.; Radu, F.; Rink, K.; Shao, H.; Shao, H.; Sun, F.; Sun, Y.; Singh, A.; Taron, J.; Walther, M.; Wang, W.; Watanabe, N.; Wu, Y.; Xie, M.; Xu, W.; Zehner, B., Opengeosys: an open-source initiative for numerical simulation of thermo-hydro-mechanical/chemical (thm/c) processes in porous media, Environ Earth Sci, 67, 2, 589-599 (2012) · doi:10.1007/s12665-012-1546-x
[28] Kotakemori H, Hasegawa H, Kajiyama T, Nukada A, Suda R, Nishida A (2008) Performance evaluation of parallel sparse matrix-vector products on sgi altix3700. In: Mueller M, Chapman B, de Supinski B, Malony A, Voss M (eds) OpenMP shared memory parallel programming, volume 4315 of lecture notes in computer science
[29] Kotakemori H, Hasegawa H, Nishida A (2005) Performance evaluation of a parallel iterative method library using openmp. In: Proceedings of the 8th international conference on high performance computing in Asia Pacific Region (HPC Asia 2005). IEEE
[30] Lichtner PC, Hammond GE, Lu C, Karra S, Bisht G, Andre B, Mills RT, Kumar J, Frederick JM (2018a) PFLOTRAN User Manual. http://www.documentation.pflotran.org
[31] Lichtner PC, Hammond GE, Lu C, Karra S, Bisht G, Andre B, Mills RT, Kumar J, Frederick JM (2018b) PFLOTRAN Web page. http://www.pflotran.org
[32] Marty, NCM; Bildstein, O.; Blanc, P.; Claret, F.; Cochepin, B.; Gaucher, EC; Jacques, D.; Lartigue, JE; Liu, S.; Meeussen, KUMJCL; Munier, I.; Pointeau, I.; Su, D.; Steefel, CI, Benchmarks for multicomponent reactive transport across a cement/clay interface, Comput Geosci, 19, 3, 635-653 (2015) · doi:10.1007/s10596-014-9463-6
[33] Mavriplis, DJ, Unstructured grid techniques, Annu Rev Fluid Mech, 29, 473-514 (1997) · doi:10.1146/annurev.fluid.29.1.473
[34] May DA, Sanan P, Rupp K, Knepley M, Smith B (2016) Extreme-scale multigrid components within petsc. In: PASC ’16: Proceedings of the platform for advanced scientific computing conference, association for computing machinery, New York, NY, United States, pp 1-12
[35] Mayer KU (1999) A numerical model for multicomponent reactive transport in variably saturated porous media. Ph.D. thesis, University of Waterloo
[36] Mayer, K.; MacQuarrie, K., Solution of the momas reactive transport benchmark with min3p- model formulation and simulation results, Comput Geosci, 14, 3, 405-419 (2010) · Zbl 1426.76690 · doi:10.1007/s10596-009-9158-6
[37] Mayer, KU; Frind, EO; Blowes, DW, Multicomponent reactive transport modeling in variably saturated porous media using a generalized formulation for kinetically controlled reactions, Water Resour Res, 38, 9, 1174 (2002) · doi:10.1029/2001WR000862
[38] Mayer K, Benner S, Frind E, Thornton S, Lerner D (2001) Reactive transport modeling of processes controlling the distribution and natural attenuation of phenolic compounds in a deep sandstone aquifer. J Contam Hydrol 53(3):341-368
[39] Meeussen, JCL, Orchestra: an object-oriented framework for implementing chemical equilibrium models, Environ Sci Technol, 37, 6, 1175-1182 (2003) · doi:10.1021/es025597s
[40] Mostaghimi, P.; Liu, M.; Arns, CH, Numerical simulation of reactive transport on micro-ct images, Math Geosci, 48, 8, 963-983 (2016) · doi:10.1007/s11004-016-9640-3
[41] Nishida A (2010) Experience in developing an open source scalable software infrastructure in Japan. In: Taniar D, Gervasi O, Murgante B, Pardede E, Apduhan B (eds) Computational science and its applications—ICCSA 2010, volume 6017 of lecture notes in computer science. Springer, Berlin
[42] Nishikawa, H., First-, second-, and third-order finite-volume schemes for diffusion, J Comput Phys, 256, 791-805 (2014) · Zbl 1349.65380 · doi:10.1016/j.jcp.2013.09.024
[43] Nishikawa, H., First, second, and third order finite-volume schemes for advection-diffusion, J Comput Phys, 273, 15, 287-309 (2014) · Zbl 1352.65283 · doi:10.1016/j.jcp.2014.05.021
[44] Parkhurst DL, Appelo C (2013) Description of input and examples for PHREEQC version 3-a computer program for speciation, batch-reaction, one-dimensional transport, and inverse geochemical calculations. Technical report, U.S. Geological Survey. https://pubs.usgs.gov/tm/06/a43/
[45] Pasdunkorale, J.; Turner, IW, A second order control-volume finite-element least-squares strategy for simulating diffusion in strongly anisotropic media, J Comput Math, 23, 1, 1-16 (2005) · Zbl 1309.74074
[46] Perko, J.; Mayer, KU; Kosakowski, G.; Windt, LD; Govaerts, J.; Jacques, D.; Su, D.; Meeussen, JCL, Decalcification of cracked cement structures, Comput Geosci, 19, 3, 673-693 (2015) · doi:10.1007/s10596-014-9467-2
[47] Samper J, Yang C, Zheng L, Montenegro L, Xu T, Dai, Zhang G, Lu C, Moreira S (2012) CORE2D V4: a code for water flow, heat and solute transport, geochemical reactions, and microbial processes. In: Chapter groundwater reactive transport models chapter 7. Bentham Science Publishers, pp 160-185
[48] Sejekan CB (2016) Improving finite-volume diffusive fluxes through better reconstruction. Master’s thesis, University of British Columbia · Zbl 1390.76514
[49] Şengör, SS; Mayer, KU; Greskowiak, J.; Wanner, C.; Su, D.; Prommer, H., A reactive transport benchmark on modeling biogenic uraninite re-oxidation by fe(iii)-(hydr)oxides, Comput Geosci, 19, 3, 569-583 (2015) · doi:10.1007/s10596-015-9480-0
[50] Simmons, CT; Elder, JW, The elder problem, Groundwater, 55, 6, 926-930 (2017) · doi:10.1111/gwat.12593
[51] Simpson, M.; Clement, T., Theoretical analysis of the worthiness of henry and elder problems as benchmarks of density-dependent groundwater flow models, Adv Water Resour, 26, 1, 17-31 (2003) · doi:10.1016/S0309-1708(02)00085-4
[52] Šimůnek J, Jacques D, Šejna M, van Genuchten MT (2012) The hp2 program for hydrus (2d/3d) a coupled code for simulating two-dimensional variablysaturated water flow, head transport, solute transport and biogeochemistry in porous media (hydrus + phreeqc + 2d) version 1.0. Technical report, University of California Riverside
[53] Steefel CI, Molins S (2016) Crunchflow: Software for modeling multicomponent reactive flow and transport user’s manual. Technical report, Lawrence Berkeley National Laboratory. http://www.csteefel.com/CrunchFlowManual.pdf
[54] Steefel, CI; Appelo, CAJ; Arora, B.; Jacques, D.; Kalbacher, T.; Kolditz, O.; Lagneau, V.; Lichtner, PC; Mayer, KU; Meeussen, JCL; Molins, S.; Moulton, D.; Shao, H.; Simunek, J.; Spycher, N.; Yabusaki, SB; Yeh, GT, Reactive transport codes for subsurface environmental simulation, Comput Geosci, 19, 445-478 (2014) · Zbl 1323.86002 · doi:10.1007/s10596-014-9443-x
[55] Su, D.; Mayer, KU; MacQuarrie, KTB, Parallelization of min3p-THCM: a high performance computational framework for subsurface flow and reactive transport simulation, Environ Model Softw, 95, 271-289 (2017) · doi:10.1016/j.envsoft.2017.06.008
[56] Su D, Mayer KU, MacQuarrie KTB (2020) Numerical investigation of flow instabilities using fully unstructured discretization for variably saturated flow problems. Adv Water Resour (in press)
[57] Trinchero, P.; Molinero, J.; Ebrahimi, H.; Puigdomenech, I.; Gylling, B.; Svensson, U.; Bosbach, D.; Deissmann, G., Simulating oxygen intrusion into highly heterogeneous fractured media using high performance computing, Math Geosci, 50, 5, 549-567 (2018) · doi:10.1007/s11004-017-9718-6
[58] van der Lee, J.; De Windt, L.; Lagneau, V.; Goblet, P., Module-oriented modeling of reactive transport with hytec, Comput Geosci, 29, 3, 265-275 (2003) · doi:10.1016/S0098-3004(03)00004-9
[59] Voss, CI; Souza, WR, Variable density flow and solute transport simulation of regional aquifers containing a narrow freshwater-saltwater transition zone, Water Resour Res, 23, 10, 1851-1866 (1987) · doi:10.1029/WR023i010p01851
[60] Wei, X.; Li, W.; Tian, H.; Li, H.; Xu, H.; Xu, T., Thc-mp: High performance numerical simulation of reactive transport and multiphase flow in porous media, Comput Geosci, 80, 26-37 (2015) · doi:10.1016/j.cageo.2015.03.014
[61] Wexler EJ (1992) Techniques of water-resources investigations of the united states geologic survey. In: Chapter b7—analytical solutions for one-, two-, and three-dimensional solute transport in ground-water systems with uniform flow. Technical report, USGS
[62] Wheeler MF, Sun S, Thomas SG (2012) Modeling of flow and reactive transport in IPARS. In: Chapter groundwater reactive transport models chapter 2. Bentham Science Publishers, pp 42-73
[63] White MD, Oostrom M (2006) Stomp subsurface transport over multiple phases version 4.0 user’s guide. Technical report, Pacific Northwest National Laboratory
[64] Wösten, JHM; van Genuchten, MT, Using texture and other soil properties to predict the unsaturated soil hydraulic functions, Soil Sci Soc Am J, 52, 1762-1770 (1988) · doi:10.2136/sssaj1988.03615995005200060045x
[65] Xu T, Spycher N, Sonnenthal E, Zheng L, Pruess K (2012) Toughreact user’s guide: A simulation program for non-isothermal multiphase reactive transport in variably saturated geologic media,version 2.0. Technical report, Lawrence Berkeley National Laboratory
[66] Yeh GTG, Sun J, Jardine PM, Burgos WD, Fang Y, Li MH, Siegel MD (2004) Hydrogeochem 5.0: A three-dimensional model of coupled fluid flow, thermal transport, and hydrogeochemical transport through variably saturated conditions—version 5.0. Technical report, Oak Ridge national Laboratory
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.