Emulated multivariate global sensitivity analysis for complex computer models applied to agricultural simulators. (English) Zbl 1426.62342

Summary: Complex mechanistic computer models often produce functional or multivariate output. Sensitivity analysis can be used to determine what input parameters are responsible for uncertainty in the output. Much of the literature around sensitivity analysis has focused on univariate output. Recent advances have been made in sensitivity analysis for multivariate output. However, these methods often depend on a significant number of model runs and may still be computationally intensive for practical purposes. Emulators have been a proven method for reducing the required number of model runs for univariate sensitivity analysis, with some recent development for multivariate computer models. We propose the use of generalized additive models and random forests combined with a principal component analysis for emulation for a multivariate sensitivity analysis. We demonstrate our method using a complex agricultural simulators.


62P12 Applications of statistics to environmental and related topics
Full Text: DOI


[1] Adams, B., Ebeida, M., Eldred, M., Jakeman, J., Swiler, L., Stephens, J., Vigil, D., Wildey, T., Bohnhoff, W., Eddy, J., Hu, K., Bauman, L., and Hough, P. (2014). Dakota, a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis. Technical report, Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States).
[2] Asseng, S.; Ewert, F.; Rosenzweig, C.; Jones, J.; Hatfield, J.; Ruane, A.; Boote, K.; Thorburn, P.; Rtter, R.; Cammarano, D.; Brisson, N.; Basso, B.; Martre, P.; Aggarwal, P.; Angulo, C.; Bertuzzi, P.; Biernath, C.; Challinor, A.; Doltra, J.; Gayler, S.; Goldberg, R.; Grant, R.; Heng, L.; Hooker, J.; Hunt, L.; Ingwersen, J.; Izaurralde, R.; Kersebaum, K.; Mller, C.; Naresh Kumar, S.; Nendel, C.; O’Leary, G.; Olesen, J.; Osborne, T.; Palosuo, T.; Priesack, E.; Ripoche, D.; Semenov, M.; Shcherbak, I.; Steduto, P.; Stckle, C.; Stratonovitch, P.; Streck, T.; Supit, I.; Tao, F.; Travasso, M.; Waha, K.; Wallach, D.; White, J.; Williams, J.; Wolf, J., Uncertainty in simulating wheat yields under climate change, Nature Climate Change, 3, 827-832, (2013)
[3] Bastos, L.; O’Hagan, A., Diagnostics for gaussian process emulators, Technometrics, 51, 425-438, (2009)
[4] Bayarri, M.; Berger, J.; Cafeo, J.; Garcia-Donato, G.; Liu, F.; Palomo, J.; Parthasarathy, R.; Paulo, R.; Sacks, J.; Walsh, D., Computer model validation with functional output, Annals of Statistics, 35, 1874-1906, (2007) · Zbl 1144.62368
[5] Breiman, L., Random forests, Machine Learning, 45, 5-32, (2001) · Zbl 1007.68152
[6] Campbell, K.; McKay, M.; Williams, B., Sensitivity analysis when model outputs are functions, Reliability Engineering and System Safety, 91, 1468-1472, (2006)
[7] Carnell, R. (2016). lhs: Latin Hypercube Samples. R package version 0.14.
[8] Cressie, N. and Wikle, C. K. (2011). Statistics for spatio-temporal data. John Wiley & Sons. · Zbl 1273.62017
[9] Veiga, S., Global sensitivity analysis with dependence measures, Journal of Statistical Computation and Simulation, 85, 1283-1305, (2015)
[10] Lozzo, M.; Marrel, A., Sensitivity analysis with dependence and variance-based measures for spatio-temporal numerical simulators, Stochastic Environmental Research and Risk Assessment, 31, 1437-1453, (2017)
[11] Duchon, J.; Schempp, W. (ed.); Zeller, K. (ed.), Splines minimizing rotation-invariant semi-norms in sobolev spaces, 85-100, (1977), Berlin, Heidelberg
[12] Gamboa, F.; Janon, A.; Klein, T.; Lagnoux, A., Sensitivity indices for multivariate outputs, Comptes Rendus Mathematique, 351, 307-310, (2013) · Zbl 06186984
[13] Gamboa, F.; Janon, A.; Klein, T.; Lagnoux, A., Sensitivity analysis for multidimensional and functional outputs, Electronic Journal of Statistics, 8, 575-603, (2014) · Zbl 1348.62106
[14] Garcia-Cabrejo, O.; Valocchi, A., Global sensitivity analysis for multivariate output using polynomial chaos expansion, Reliability Engineering and System Safety, 126, 25-36, (2014)
[15] Gladish, D.; Pagendam, D.; Peeters, L.; Kuhnert, P.; Vaze, J., Emulation engines: Choice and quantification of uncertainty for complex hydrological models, Journal of Agricultural, Biological, and Environmental Statistics, 23, 39-62, (2018) · Zbl 1391.62272
[16] Gretton, A.; Bousquet, O.; Smola, A.; Schölkopf, B.; Jain, S. (ed.); Simon, H. (ed.); Tomita, E. (ed.), Measuring statistical dependence with hilbert-schmidt norms, 63-77, (2005), Berlin, Heidelberg · Zbl 1168.62354
[17] Gupta, H. and Razavi, S. (2016). Challenges and Future Outlook of Sensitivity Analysis.
[18] Hamby, D., A review of techniques for parameter sensitivity analysis of environmental models, Environmental Monitoring and Assessment, 32, 135-154, (1994)
[19] Higdon, D.; Gattiker, J.; Williams, B.; Rightley, M., Computer model calibration using high-dimensional output, Journal of the American Statistical Association, 103, 570-583, (2008) · Zbl 1469.62414
[20] Hochman, Z.; Prestwidge, D.; Carberry, P., Crop sequences in australia’s northern grain zone are less agronomically efficient than implied by the sum of their parts, Agricultural Systems, 129, 124-132, (2014)
[21] Holzworth, D.; Huth, N.; deVoil, P.; Zurcher, E.; Herrmann, N.; McLean, G.; Chenu, K.; Oosterom, E.; Snow, V.; Murphy, C.; Moore, A.; Brown, H.; Whish, J.; Verrall, S.; Fainges, J.; Bell, L.; Peake, A.; Poulton, P.; Hochman, Z.; Thorburn, P.; Gaydon, D.; Dalgliesh, N.; Rodriguez, D.; Cox, H.; Chapman, S.; Doherty, A.; Teixeira, E.; Sharp, J.; Cichota, R.; Vogeler, I.; Li, F.; Wang, E.; Hammer, G.; Robertson, M.; Dimes, J.; Whitbread, A.; Hunt, J.; Rees, H.; McClelland, T.; Carberry, P.; Hargreaves, J.; MacLeod, N.; McDonald, C.; Harsdorf, J.; Wedgwood, S.; Keating, B., Apsim - evolution towards a new generation of agricultural systems simulation, Environmental Modelling and Software, 62, 327-350, (2014)
[22] Holzworth, D.; Snow, V.; Janssen, S.; Athanasiadis, I.; Donatelli, M.; Hoogenboom, G.; White, J.; Thorburn, P., Agricultural production systems modelling and software: Current status and future prospects, Environmental Modelling and Software, 72, 276-286, (2015)
[23] Hooten, M.; Leeds, W.; Fiechter, J.; Wikle, C., Assessing first-order emulator inference for physical parameters in nonlinear mechanistic models, Journal of Agricultural, Biological, and Environmental Statistics, 16, 475-494, (2011) · Zbl 1306.62290
[24] Jansen, M., Analysis of variance designs for model output, Computer Physics Communications, 117, 35-43, (1999) · Zbl 1015.68218
[25] Keating, B.; Carberry, P.; Hammer, G.; Probert, M.; Robertson, M.; Holzworth, D.; Huth, N.; Hargreaves, J.; Meinke, H.; Hochman, Z.; McLean, G.; Verburg, K.; Snow, V.; Dimes, J.; Silburn, M.; Wang, E.; Brown, S.; Bristow, K.; Asseng, S.; Chapman, S.; McCown, R.; Freebairn, D.; Smith, C., An overview of apsim, a model designed for farming systems simulation, European Journal of Agronomy, 18, 267-288, (2003)
[26] Lamboni, M.; Makowski, D.; Lehuger, S.; Gabrielle, B.; Monod, H., Multivariate global sensitivity analysis for dynamic crop models, Field Crops Research, 113, 312-320, (2009)
[27] Lamboni, M.; Monod, H.; Makowski, D., Multivariate sensitivity analysis to measure global contribution of input factors in dynamic models, Reliability Engineering and System Safety, 96, 450-459, (2011)
[28] Leeds, W.; Wikle, C.; Fiechter, J.; Brown, J.; Milliff, R., Modeling 3-d spatio-temporal biogeochemical processes with a forest of 1-d statistical emulators, Environmetrics, 24, 1-12, (2013)
[29] Li, L.; Lu, Z.; Wu, D., A new kind of sensitivity index for multivariate output, Reliability Engineering and System Safety, 147, 123-131, (2016)
[30] Liaw, A.; Wiener, M., Classification and regression by randomforest, R News, 2, 18-22, (2002)
[31] Makowski, D.; Naud, C.; Jeuffroy, M-H; Barbottin, A.; Monod, H., Global sensitivity analysis for calculating the contribution of genetic parameters to the variance of crop model prediction, Reliability Engineering and System Safety, 91, 1142-1147, (2006)
[32] Mara, T.; Joseph, O., Comparison of some efficient methods to evaluate the main effect of computer model factors, Journal of Statistical Computation and Simulation, 78, 167-178, (2008) · Zbl 1136.62416
[33] Marrel, A.; Iooss, B.; Jullien, M.; Laurent, B.; Volkova, E., Global sensitivity analysis for models with spatially dependent outputs, Environmetrics, 22, 383-397, (2011)
[34] Morris, M., Factorial sampling plans for preliminary computational experiments, Technometrics, 33, 161-174, (1991)
[35] Oakley, J.; O’Hagan, A., Probabilistic sensitivity analysis of complex models: A Bayesian approach, Journal of the Royal Statistical Society. Series B: Statistical Methodology, 66, 751-769, (2004) · Zbl 1046.62027
[36] Probert, M.; Dimes, J.; Keating, B.; Dalal, R.; Strong, W., Apsim’s water and nitrogen modules and simulation of the dynamics of water and nitrogen in fallow systems, Agricultural Systems, 56, 1-28, (1998)
[37] R Core Team (2017). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
[38] Saltelli, A.; Annoni, P.; Azzini, I.; Campolongo, F.; Ratto, M.; Tarantola, S., Variance based sensitivity analysis of model output. design and estimator for the total sensitivity index, Computer Physics Communications, 181, 259-270, (2010) · Zbl 1219.93116
[39] Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., and Tarantola, S. (2008). Global Sensitivity Analysis. The Primer. · Zbl 1161.00304
[40] Saltelli, A., Tarantola, S., Campolongo, F., and Ratto, M. (2004). Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. Halsted Press, New York, NY, USA. · Zbl 1049.62112
[41] Shi, J.; Wang, B.; Murray-Smith, R.; Titterington, D., Gaussian process functional regression modeling for batch data, Biometrics, 63, 714-723, (2007) · Zbl 1128.62053
[42] Sobol, I., Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates, Mathematics and Computers in Simulation, 55, 271-280, (2001) · Zbl 1005.65004
[43] Stanfill, B.; Mielenz, H.; Clifford, D.; Thorburn, P., Simple approach to emulating complex computer models for global sensitivity analysis, Environmental Modelling and Software, 74, 140-155, (2015)
[44] Stein, M., Large sample properties of simulations using latin hypercube sampling, Technometrics, 29, 143-151, (1987) · Zbl 0627.62010
[45] Storlie, C.; Helton, J., Multiple predictor smoothing methods for sensitivity analysis: Description of techniques, Reliability Engineering and System Safety, 93, 28-54, (2008)
[46] Storlie, C.; Swiler, L.; Helton, J.; Sallaberry, C., Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models, Reliability Engineering and System Safety, 94, 1735-1763, (2009)
[47] Strong, M.; Oakley, J.; Brennan, A., Estimating multiparameter partial expected value of perfect information from a probabilistic sensitivity analysis sample: A nonparametric regression approach, Medical Decision Making, 34, 311-326, (2014)
[48] Sudret, B., Global sensitivity analysis using polynomial chaos expansions, Reliability Engineering and System Safety, 93, 964-979, (2008)
[49] Wallach, D.; Goffinet, B.; Bergez, J-E; Debaeke, P.; Leenhardt, D.; Aubertot, J-N, Parameter estimation for crop models: A new approach and application to a corn model, Agronomy Journal, 93, 757-766, (2001)
[50] Wood, S. N. (2006). Generalized additive models: an introduction with R. CRC press. · Zbl 1087.62082
[51] Xiao, S., Lu, Z., and Wang, P. (2017). Multivariate global sensitivity analysis for dynamic models based on energy distance. Structural and Multidisciplinary Optimization.
[52] Xiao, S.; Lu, Z.; Wang, P., Multivariate global sensitivity analysis for dynamic models based on wavelet analysis, Reliability Engineering and System Safety, 170, 20-30, (2018)
[53] Zhao, G.; Bryan, BA; Song, X., Sensitivity and uncertainty analysis of the apsim-wheat model: Interactions between cultivar, environmental, and management parameters, Ecological Modelling, 279, 1-11, (2014)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.