ANOVA Gaussian process modeling for high-dimensional stochastic computational models. (English) Zbl 1437.62286

Chen, Chen; Liao, Qifeng

J. Comput. Phys. 416, Article ID 109519, 16 p. (2020).

Summary: In this paper we present a novel analysis of variance Gaussian process (ANOVA-GP) emulator for models governed by partial differential equations (PDEs) with high-dimensional random inputs. The Gaussian process (GP) is a widely used surrogate modeling strategy, but it can become invalid when the inputs are high-dimensional. In this new ANOVA-GP strategy, high-dimensional inputs are decomposed into unions of local low-dimensional inputs, and principal component analysis (PCA) is applied to provide dimension reduction for each ANOVA term. We then systematically build local GP models for PCA coefficients based on ANOVA decomposition to provide an emulator for the overall high-dimensional problem. We present a general mathematical framework of ANOVA-GP, validate its accuracy and demonstrate its efficiency with numerical experiments.

MSC:

62J10	Analysis of variance and covariance (ANOVA)
65C20	Probabilistic models, generic numerical methods in probability and statistics
60G15	Gaussian processes
35R60	PDEs with randomness, stochastic partial differential equations
76M35	Stochastic analysis applied to problems in fluid mechanics
76D07	Stokes and related (Oseen, etc.) flows

Keywords:

adaptive ANOVA; Gaussian process; model reduction; uncertainty quantification

Software:

Sparse Grid Interpolation; GPML; gss; IFISS

References:

[1]	Ainsworth, M.; Oden, J., A Posteriori Error Estimation in Finite Element Analysis (2000), Wiley: Wiley New York · Zbl 1008.65076
[2]	Elman, H.; Silvester, D.; Wathen, A., Finite Elements and Fast Iterative Solvers: with Applications in Incompressible Fluid Dynamics (2014), Oxford University Press (UK) · Zbl 1304.76002
[3]	Kennedy, M. C.; O’Hagan, A., Bayesian calibration of computer models, J. R. Stat. Soc., Ser. B, Stat. Methodol., 63, 3, 425-464 (2001) · Zbl 1007.62021
[4]	Oakley, J.; O’Hagan, A., Bayesian inference for the uncertainty distribution of computer model outputs, Biometrika, 89, 769-784 (2002)
[5]	Kennedy, M.; Anderson, C.; Conti, S.; O’Hagan, A., Case studies in Gaussian process modelling of computer codes, Reliab. Eng. Syst. Saf., 91, 1301-1309 (2006)
[6]	Tagade, P. M.; Jeong, B.-M.; Choi, H.-L., A Gaussian process emulator approach for rapid contaminant characterization with an integrated multizone-CFD model, Build. Environ., 70, 232-244 (2013)
[7]	Ghanem, R. G.; Spanos, P. D., Stochastic Finite Elements: A Spectral Approach (2003), Courier Corporation
[8]	Xiu, D.; Karniadakis, G. E., The Wiener-Askey polynomial chaos for stochastic differential equations, SIAM J. Sci. Comput., 24, 2, 619-644 (2002) · Zbl 1014.65004
[9]	Xiu, D.; Hesthaven, J., High-order collocation methods for differential equations with random inputs, SIAM J. Sci. Comput., 27, 1118-1139 (2005) · Zbl 1091.65006
[10]	Xiu, D., Numerical Methods for Stochastic Computations: A Spectral Method Approach (2010), Princeton University Press: Princeton University Press Princeton · Zbl 1210.65002
[11]	Boyaval, S.; Bris, C. L.; Lelièvre, T.; Maday, Y.; Nguyen, N.; Patera, A., Reduced basis techniques for stochastic problems, Arch. Comput. Methods Eng., 17, 1-20 (2010) · Zbl 1269.65005
[12]	Elman, H.; Liao, Q., Reduced basis collocation methods for partial differential equations with random coefficients, SIAM/ASA J. Uncertain. Quantificat., 1, 192-217 (2013) · Zbl 1282.35424
[13]	Chen, P.; Quarteroni, A.; Rozza, G., Comparison between reduced basis and stochastic collocation methods for elliptic problems, J. Sci. Comput., 59, 187-216 (2014) · Zbl 1301.65007
[14]	Jiang, J.; Chen, Y.; Narayan, A., A goal-oriented reduced basis methods-accelerated generalized polynomial chaos algorithm, SIAM/ASA J. Uncertain. Quantificat., 4, 1398-1420 (2016) · Zbl 1352.65012
[15]	Higdon, D.; Gattiker, J.; Williams, B.; Rightley, M., Computer model calibration using high-dimensional output, J. Am. Stat. Assoc., 103, 482, 570-583 (2008) · Zbl 1469.62414
[16]	Ma, X.; Zabaras, N., Kernel principal component analysis for stochastic input model generation, J. Comput. Phys., 230, 7311-7331 (2011) · Zbl 1252.65014
[17]	Xing, W.; Triantafyllidis, V.; Shah, A.; Nair, P.; Zabaras, N., Manifold learning for the emulation of spatial fields from computational models, J. Comput. Phys., 326, 666-690 (2016) · Zbl 1373.68340
[18]	Guo, M.; Hesthaven, J. S., Reduced order modeling for nonlinear structural analysis using Gaussian process regression, Comput. Methods Appl. Mech. Eng., 341, 807-826 (2018) · Zbl 1440.65206
[19]	Tripathy, R.; Bilionis, I.; Gonzalez, M., Gaussian processes with built-in dimensionality reduction: applications to high-dimensional uncertainty propagation, J. Comput. Phys., 321, 191-223 (2016) · Zbl 1349.65049
[20]	Storlie, C. B.; Lane, W. A.; Ryan, E. M.; Gattiker, J. R.; Higdon, D. M., Calibration of computational models with categorical parameters and correlated outputs via Bayesian smoothing spline ANOVA, J. Am. Stat. Assoc., 110, 509, 68-82 (2015)
[21]	Rabitz, H.; Aliş, Ö. F.; Shorter, J.; Shim, K., Efficient input-output model representations, Comput. Phys. Commun., 117, 1-2, 11-20 (1999) · Zbl 1015.68219
[22]	Rabitz, H.; Aliş, Ö. F., General foundations of high-dimensional model representations, J. Math. Chem., 25, 2-3, 197-233 (1999) · Zbl 0957.93004
[23]	Gao, Z.; Hesthaven, J. S., On ANOVA expansions and strategies for choosing the anchor point, Appl. Math. Comput., 217, 7, 3274-3285 (2010) · Zbl 1206.65125
[24]	Ma, X.; Zabaras, N., An adaptive high-dimensional stochastic model representation technique for the solution of stochastic partial differential equations, J. Comput. Phys., 229, 10, 3884-3915 (2010) · Zbl 1189.65019
[25]	Zhang, Z.; Choi, M.; Karniadakis, G., Error estimates for the ANOVA method with polynomial chaos interpolation: tensor product functions, SIAM J. Sci. Comput., 34, 2, A1165-A1186 (2012) · Zbl 1248.65031
[26]	Yang, X.; Choi, M.; Lin, G.; Karniadakis, G. E., Adaptive ANOVA decomposition of stochastic incompressible and compressible flows, J. Comput. Phys., 231, 4, 1587-1614 (2012) · Zbl 1408.76428
[27]	Hesthaven, J. S.; Zhang, S., On the use of ANOVA expansions in reduced basis methods for parametric partial differential equations, J. Sci. Comput., 69, 1, 292-313 (2016) · Zbl 1352.65508
[28]	Sobol, I. M., Theorems and examples on high dimensional model representation, Reliab. Eng. Syst. Saf., 79, 2, 187-193 (2003)
[29]	Wang, X., On the approximation error in high dimensional model representation, (Simulation Conference, 2008. WSC 2008, Winter (2008), IEEE), 453-462
[30]	Xu, H.; Rahman, S., A generalized dimension-reduction method for multidimensional integration in stochastic mechanics, Int. J. Numer. Methods Eng., 61, 12, 1992-2019 (2004) · Zbl 1075.74707
[31]	Novak, E.; Ritter, K., High dimensional integration of smooth functions over cubes, Numer. Math., 75, 1, 79-97 (1996) · Zbl 0883.65016
[32]	Trefethen, L. N., Is Gauss quadrature better than clenshaw-curtis?, SIAM Rev., 50, 1, 67-87 (2008) · Zbl 1141.65018
[33]	Jolliffe, I., Principal component analysis, (International Encyclopedia of Statistical Science (2011), Springer), 1094-1096
[34]	Vidal, R.; Ma, Y.; Sastry, S. S., Generalized Principal Component Analysis, vol. 5 (2016), Springer · Zbl 1349.62006
[35]	Holmes, P.; Lumley, J. L.; Berkooz, G., Turbulence, Coherent Structures, Dynamical Systems and Symmetry (1996), Cambridge, New York · Zbl 0890.76001
[36]	Benner, P.; Gugercin, S.; Willcox, K., A survey of model reduction methods for parametric systems, SIAM Rev., 57, 483-531 (2015) · Zbl 1339.37089
[37]	Rasmussen, C. E., Gaussian processes in machine learning, (Advanced Lectures on Machine Learning (2004), Springer), 63-71 · Zbl 1120.68436
[38]	Andrianakis, I.; Challenor, P. G., The effect of the nugget on Gaussian process emulators of computer models, Comput. Stat. Data Anal., 56, 12, 4215-4228 (2012) · Zbl 1255.62306
[39]	Snelson, E. L., Flexible and efficient Gaussian process models for machine learning (2007), UCL (University College London), PhD thesis
[40]	Rasmussen, C. E.; Nickisch, H., Gaussian processes for machine learning (gpml) toolbox, J. Mach. Learn. Res., 11, 3011-3015 (Nov. 2010)
[41]	Møller, M. F., A scaled conjugate gradient algorithm for fast supervised learning, Neural Netw., 6, 4, 525-533 (1993)
[42]	Liao, Q.; Lin, G., Reduced basis ANOVA methods for partial differential equations with high-dimensional random inputs, J. Comput. Phys., 317, 148-164 (2016) · Zbl 1349.65684
[43]	Cho, H.; Elman, H. C., An adaptive reduced basis collocation method based on PCM ANOVA decomposition for anisotropic stochastic PDEs, Int. J. Uncertain. Quantificat., 8, 193-210 (2018)
[44]	Braess, D., Finite Elements: Theory, Fast Solvers, and Applications in Solid Mechanics (2007), Cambridge University Press · Zbl 1118.65117
[45]	Klimke, A., Sparse Grid Interpolation Toolbox – user’s guide (2007), University of Stuttgart, Tech. Rep. IANS report 2007/017
[46]	Tang, K.; Congedo, P. M.; Abgrall, R., Sensitivity analysis using anchored ANOVA expansion and high-order moments computation, Int. J. Numer. Methods Eng., 102, 1554-1584 (2015) · Zbl 1352.62120
[47]	Bengio, Y.; Delalleau, O.; Roux, N. L., The curse of highly variable functions for local kernel machines, (Weiss, Y.; Schölkopf, B.; Platt, J. C., Advances in Neural Information Processing Systems, vol. 18 (2006), MIT Press), 107-114
[48]	Brown, J., Mean square truncation error in series expansions of random functions, J. Soc. Ind. Appl. Math., 8, 1, 28-32 (1960) · Zbl 0094.12205
[49]	Schwab, C.; Todor, R. A., Karhunen-Loève approximation of random fields by generalized fast multipole methods, J. Comput. Phys., 217, 1, 100-122 (2006) · Zbl 1104.65008
[50]	Powell, C.; Elman, H., Block-diagonal preconditioning for spectral stochastic finite-element systems, IMA J. Numer. Anal., 29, 350-375 (2009) · Zbl 1169.65007
[51]	Elman, H. C.; Ramage, A.; Silvester, D. J., IFISS: A computational laboratory for investigating incompressible flow problems, SIAM Rev., 56, 2, 261-273 (2014) · Zbl 1426.76645