Hostname: page-component-78c5997874-xbtfd Total loading time: 0 Render date: 2024-11-10T17:48:34.717Z Has data issue: false hasContentIssue false
  • English
  • Français

Toward Cost-Effective Reservoir Simulation Solvers on GPUs

Part of: Optics, electromagnetic theory - General

Published online by Cambridge University Press:  19 September 2016

Zheng Li ,
Shuhong Wu ,
Jinchao Xu  and
Chensong Zhang
Zheng Li*
Affiliation:
Kunming University of Science and Technology, Kunming 650093, China
Shuhong Wu*
Affiliation:
Research Institute of Petroleum Exploration and Development, CNPC, Beijing 100083, China
Jinchao Xu*
Affiliation:
Department of Mathematics, Penn State University, University Park, PA 16802, USA
Chensong Zhang*
Affiliation:
LSEC & NCMIS, Academy of Mathematics and Systems Science, Beijing 100190, China
*
*Corresponding author. Email:lizhxtu@126.com (Z. Li), wush@petrochina.com.cn (S.Wu), xu@math.psu.edu (J. Xu), zhangcs@lsec.cc.ac.cn (C. Zhang)
*Corresponding author. Email:lizhxtu@126.com (Z. Li), wush@petrochina.com.cn (S.Wu), xu@math.psu.edu (J. Xu), zhangcs@lsec.cc.ac.cn (C. Zhang)
*Corresponding author. Email:lizhxtu@126.com (Z. Li), wush@petrochina.com.cn (S.Wu), xu@math.psu.edu (J. Xu), zhangcs@lsec.cc.ac.cn (C. Zhang)
*Corresponding author. Email:lizhxtu@126.com (Z. Li), wush@petrochina.com.cn (S.Wu), xu@math.psu.edu (J. Xu), zhangcs@lsec.cc.ac.cn (C. Zhang)
Get access

Abstract

In this paper, we focus on graphical processing unit (GPU) and discuss how its architecture affects the choice of algorithm and implementation of fully-implicit petroleum reservoir simulation. In order to obtain satisfactory performance on new many-core architectures such as GPUs, the simulator developers must know a great deal on the specific hardware and spend a lot of time on fine tuning the code. Porting a large petroleum reservoir simulator to emerging hardware architectures is expensive and risky. We analyze major components of an in-house reservoir simulator and investigate how to port them to GPUs in a cost-effective way. Preliminary numerical experiments show that our GPU-based simulator is robust and effective. More importantly, these numerical results clearly identify the main bottlenecks to obtain ideal speedup on GPUs and possibly other many-core architectures.

Keywords

GPUsreservoir simulationfully-implicit method

MSC classification

Secondary: 78A48: Composite media; random media
Type
Research Article
Information
Advances in Applied Mathematics and Mechanics , Volume 8 , Issue 6 , December 2016 , pp. 971 - 991
Copyright
Copyright © Global-Science Press 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Appleyard, J. R., Appleyard, J. D., Wakefield, M. A. and Desitter, A. L., Accelerating reservoir simulators using GPU technology, SPE Reservoir Simulation Symposium, 2011.CrossRefGoogle Scholar
[2] Baker, A. H., Falgout, R. D., Kolev, T. V. and Yang, U. M., Multigrid smoothers for ultraparallel computing, SIAM J. Sci. Comput., 33(5) (2011), pp. 2864–2887.CrossRefGoogle Scholar
[3] Bank, R. E., Chan, T. F., Coughran, W. M. Jr and Smith, R. K., The Alternate-Block-Factorization procedure for systems of partial differential equations, BIT Numer. Math., 29(4) (1989), pp. 938–954.CrossRefGoogle Scholar
[4] Bell, N., Dalton, S. and Olson, L. N., Exposing fine-grained parallelism in algebraic multigrid methods, SIAM J. Sci. Comput., 34(4) (2012), pp. C123–C152.CrossRefGoogle Scholar
[5] Bell, N. and Garland, M., Efficient sparse matrix-vector multiplication on CUDA, Technical report, Nvidia Technical Report NVR-2008-004, Nvidia Corporation, 2008.Google Scholar
[6] Braess, D., Towards algebraic multigrid for elliptic problems of second order, Computing, 55(4) (1995), pp. 379–393.CrossRefGoogle Scholar
[7] Brandt, A., Algebraic multigrid theory: the symmetric case, Appl. Math. Comput., 19(1) (1986), pp. 23–56.Google Scholar
[8] Brandt, A., McCormick, S. and Ruge, J., Algebraic multigrid (amg) for automatic multigrid solutions with application to geodetic computations, Report, Inst. for Computational Studies, Fort Collins, Colo, 1982.Google Scholar
[9] Brandt, A., McCoruick, S. and Ruge, J., Algebraic multigrid (amg) for sparse matrix equations, Sparsity Appl., (1985), pp. 257–284.Google Scholar
[10] Brannick, J., Chen, Y., Hu, X. and Zikatanov, L., Parallel unsmoothed aggregation algebraic multigrid algorithms on gpus, Numerical Solution of Partial Differential Equations: Theory, Algorithms and Their Applications, pages 81–102, Springer, 2013.CrossRefGoogle Scholar
[11] Byun, J.-H., Lin, R., Yelick, K. A. and Demmel, J., Autotuning sparse matrix-vector multiplication for multicore, Technical Report UCB/EECS-2012-215, EECS Department, University of California, Berkeley, November 2012.Google Scholar
[12] Cao, H., Tchelepi, H., Wallis, J. and Yardumian, H., Parallel scalable unstructured CPR-type linear solver for reservoir simulation, Paper SPE 96809 presented at the SPE Annual Technical Conference and Exhibition, Dallas, Texas, 9-12 October, 2005.CrossRefGoogle Scholar
[13] Chen, Z., Huan, G. and Ma, Y., Computational Methods for Multiphase Flows in Porous Media, Volume 2, SIAM, 2006.CrossRefGoogle Scholar
[14] Choi, J. W., Singh, A. and Vuduc, R. W., Model-driven autotuning of sparse matrix-vector multiply on GPUs, ACM SIGPLAN Notices, 45(5) (2010), pp. 115.CrossRefGoogle Scholar
[15] Christie, M. and Blunt, M., Tenth SPE comparative solution project: A comparison of upscaling techniques, SPE Reservoir Evaluation & Engineering, 4(04) (2001), pp. 308–317.CrossRefGoogle Scholar
[16] Coats, K. H. et al., A note on IMPES and some IMPES-based simulation models, SPE J., 5(03) (2000), pp. 245–251.CrossRefGoogle Scholar
[17] CUSOLVER, https://developer.nvidia.com/cusolver.Google Scholar
[18] Dang, H. V. and Schmidt, B., CUDA-enabled sparse matrix-vector multiplication on GPUs using atomic operations, Parallel Comput., 39(11) (2013), pp. 737–750.CrossRefGoogle Scholar
[19] Dogru, A. H., Fung, L. S. and Middya, U. et al., A next-generation parallel reservoir simulator for giant reservoirs, SPE/EAGE Reservoir Characterization & Simulation Conference, 2009.CrossRefGoogle Scholar
[20] Douglas, J. Jr, Peaceman, D. and Rachford, H. Jr et al., A method for calculating multi-dimensional immiscible displacement, Trans. Amer. Inst. Min. Metallurgical Petroleum Eng., pages 297–306, 1959.Google Scholar
[21] Esler, K., Mukundakrishnan, K., Natoli, V., Shumway, J., Zhang, Y. and Gilman, J., Realizing the potential of GPUs for reservoir simulation, ECMOR XIV-14th European Conference on the Mathematics of Oil Recovery, 2014.CrossRefGoogle Scholar
[22] Falgout, R., An introduction to algebraic multigrid computing, Comput. Sci. Eng., 8(6) (2006).CrossRefGoogle Scholar
[23] Feng, C., Multilevel Iterative Methods and Solvers for Reservoir Simulation on CPU-GPU Heterogenous Computers, PhD thesis, Xiangtan University, 2014.Google Scholar
[24] Fung, L. S., Sindi, M. O. and Dogru, A. H. et al., Multi-paradigm parallel acceleration for reservoir simulation, SPE Reservoir Simulation Symposium, 2013.CrossRefGoogle Scholar
[25] Gandham, R., Esler, K. and Zhang, Y., A GPU accelerated aggregation algebraic multigrid method, Comput. Math. Appl., 68(10) (2014), pp. 1151–1160.CrossRefGoogle Scholar
[26] Hayder, M. E. and Baddourah, M. et al., Challenges in high performance computing for reservoir simulation, Paper SPE, 152414 (2012), pp. 4–7.Google Scholar
[27] Hu, X., Vassilevski, P. S. and Xu, J., Comparative convergence analysis of nonlinear AMLI-cycle multigrid, SIAM J. Numer. Anal., 51(2) (2013), pp. 1349–1369.CrossRefGoogle Scholar
[28] Kim, H., Xu, J. and Zikatanov, L., A multigrid method based on graph matching for convection–diffusion equations, Numer. Linear Algebra Appl., 10(1-2) (2003), pp. 181–195.CrossRefGoogle Scholar
[29] Klie, H. M., Sudan, H. H., Li, R. and Saad, Y. et al., Exploiting capabilities of many core platforms in reservoir simulation, SPE Reservoir Simulation Symposium, Society of Petroleum Engineers, 2011.CrossRefGoogle Scholar
[30] Lacroix, S., Vassilevski, Y. V. and Wheeler, M. F., Decoupling preconditioners in the implicit parallel accurate reservoir simulator (IPARS), Numerical Linear Algebra Appl., 8(8) (2001), pp. 537–549.CrossRefGoogle Scholar
[31] Li, R. and Saad, Y., GPU-accelerated preconditioned iterative linear solvers, J. Supercomput., 63(2) (2013), pp. 443–466.CrossRefGoogle Scholar
[32] Liu, H., Yang, B. and Chen, Z., Accelerating algebraic multigrid solvers on NVIDIA GPUs, Comput. Math. Appl., 70(5) (2015), pp. 1162–1181.CrossRefGoogle Scholar
[33] MAGMA, http://icl.cs.utk.edu/magma/.Google Scholar
[34] MUMPS, http://mumps-solver.org.Google Scholar
[35] Napov, A. and Notay, Y., An algebraic multigrid method with guaranteed convergence rate, SIAM J. Sci. Comput., 34(2) (2012), pp. A1079–A1109.CrossRefGoogle Scholar
[36] Notay, Y., Flexible conjugate gradients, SIAM J. Sci. Comput., 22(4) (2000), pp. 1444–1460.CrossRefGoogle Scholar
[37] Notay, Y., Aggregation-based algebraic multigrid for convection-diffusion equations, SIAM J. Sci. Comput., 2012.CrossRefGoogle Scholar
[38] Pavlas, E. J. Jr et al., Fine-scale simulation of complex water encroachment in a large carbonate reservoir in saudi arabia, SPE Reservoir Evaluation & Engineering, 5(05) (2002), pp. 346–354.CrossRefGoogle Scholar
[39] Peaceman, D. W., Presentation of a horizontal well in numerical reservoir simulation, The 11th SPE Symposium on Reservoir Simulation, 1991.Google Scholar
[40] Saad, Y., Iterative methods for sparse linear systems, SIAM, 2003.CrossRefGoogle Scholar
[41] Stüben, K., Algebraic Multigrid (AMG): an Introduction with Applications, GMD Forschungszentrum Informationstechnik, 1999.Google Scholar
[42] Sudan, H., Klie, H., Li, R. and Saad, Y., High performance manycore solvers for reservoir simulation, 12th European Conference on the Mathematics of Oil Recovery, 2010.CrossRefGoogle Scholar
[43] SuperLU, http://crd-legacy.lbl.gov/~xiaoye/SuperLU/.Google Scholar
[44] Tchelepi, H. and Zhou, Y. et al., Multi-GPU parallelization of nested factorization for solving large linear systems, SPE Reservoir Simulation Symposium, Society of Petroleum Engineers, 2013.Google Scholar
[45] Trangenstein, J. A. and Bell, J. B., Mathematical structure of the black-oil model for petroleum reservoir simulation, SIAM J. Appl. Math., 49(3) (1989), pp. 749–783.CrossRefGoogle Scholar
[46] UMFPACK, http://faculty.cse.tamu.edu/davis/suitesparse.html.Google Scholar
[47] Vaněk, P., Brezina, M. and Mandel, J. et al., Convergence of algebraic multigrid based on smoothed aggregation, Numer. Math., 88(3) (2001), pp. 559–579.Google Scholar
[48] Vaněk, P., Mandel, J. and Brezina, M., Algebraic multigrid by smoothed aggregation for second and fourth order elliptic problems, Computing, 196 (1996), pp. 179–196.CrossRefGoogle Scholar
[49] Wallis, J., Incomplete Gaussian elimination as a preconditioning for generalized conjugate gradient acceleration, Paper SPE 12265 presented at the SPE Reservoir Simulation Symposium, San Francisco, California, 15-18 November, 1983.CrossRefGoogle Scholar
[50] Wallis, J., Kendall, R., Little, T. and Nolen, J., Constrained residual acceleration of conjugate residual methods, SPE, 13536 (1985), pp. 10–13.Google Scholar
[51] Wang, L., Hu, X., Cohen, J. and Xu, J., A parallel auxiliary grid algebraic multigrid method for graphic processing units, SIAM J. Sci. Comput., 35(3) (2013), pp. C263–C283.CrossRefGoogle Scholar
[52] Wu, S., Feng, C., Zhang, C.-S., Li, Q. and Al, E., A multilevel preconditioner and its shared memory implementation for new generation reservoir simulator, Petroleum Science, (2014), pp. 1–18.Google Scholar
[53] Yu, S., Liu, H., Chen, Z. J., Hsieh, B. and Shao, L. et al., GPU-based parallel reservoir simulation for large-scale simulation problems, SPE Europec/EAGE Annual Conference, Society of Petroleum Engineers, 2012.CrossRefGoogle Scholar