zbMATH — the first resource for mathematics

Regression and Kriging metamodels with their experimental designs in simulation: a review. (English) Zbl 1394.90004
Summary: This article reviews the design and analysis of simulation experiments. It focusses on analysis via two types of metamodel (surrogate. emulator), namely, low-order polynomial regression and Kriging (or Gaussian process). The metamodel type determines the design of the simulation experiment, which determines the input combinations of the simulation model. For example, a first-order polynomial regression metamodel should use a “resolution-III” design, whereas Kriging may use “Latin hypercube sampling”. More generally, polynomials of first or second order may use resolution III, IV, V, or “central composite” designs. Before applying either regression or Kriging metamodeling, the many inputs of a realistic simulation model can be screened via “sequential bifurcation”. Optimization of the simulated system may use either a sequence of low-order polynomials – known as “response surface methodology” (RSM) – or Kriging models fitted through sequential designs – including “efficient global optimization” (EGO). Finally, “robust” optimization accounts for uncertainty in some simulation inputs.

90-02 Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming
90C31 Sensitivity, stability, parametric optimization
65C60 Computational problems in statistics (MSC2010)
62-02 Research exposition (monographs, survey articles) pertaining to statistics
62L05 Sequential statistical design
62K05 Optimal statistical designs
90C26 Nonconvex programming, global optimization
Full Text: DOI
[1] Angün, M. E., Black box simulation optimization: generalized response surface methodology, CentER dissertation series, (2004), Tilburg University Tilburg, Netherlands
[2] Ankenman, B.; Nelson, B.; Staum, J., Stochastic Kriging for simulation metamodeling, Operations Research, 58, 2, 371-382, (2010) · Zbl 1342.62134
[3] Barton, R. R., Design of experiments for Fitting subsystem metamodels, (Andradóttir, S.; Healy, K. J.; Withers, D. H.; Nelson, B. L., Proceedings of the 1997 winter simulation conference, (1997)), 303-310
[4] Barton, R. R.; Meckesheimer, M., Metamodel-based simulation optimization, (Henderson, S. G.; Nelson, B. L., Chapter 18 in handbooks in operations research and management science, Vol. 13, (2006), Elsevier, Amsterdam)
[5] Bekki, J. M.; Fowler, J. W.; Mackulak, G. T.; Kulahci, M., Simulation-based cycle-time quantile estimation in manufacturing settings employing non-FIFO dispatching policies, Journal of Simulation, 3, 2, 69-128, (2009)
[6] Ben-Tal, A.; Ghaoui, L. E.; Nemirovski, A., Robust optimization, (2009), Princeton University Press, Princeton
[7] Bettonvil, B. W.M.; Castillo, E. D.; Kleijnen, J. P.C., Statistical testing of optimality conditions in multiresponse simulation-based optimization, European Journal of Operational Research, 199, 2, 448-458, (2009) · Zbl 1176.90434
[8] Bischl, B.; Mersmann, O.; Trautmann, H.; Weihs, C., Resampling methods for meta-model validation with recommendations for evolutionary computation, Evolutionary Computation, 20, 2, 249-275, (2012)
[9] Borgonovo, E.; Plischke, E., Sensitivity analysis: a review of recent advances, European Journal of Operational Research, 248, 3, 869-887, (2016) · Zbl 1346.90771
[10] Box, G. E.P.; Wilson, K. B., On the experimental attainment of optimum conditions, Journal Royal Statistical Society, Series B, 13, 1, 1-38, (1951) · Zbl 0043.34402
[11] Chang, K.-H.; Li, M.-K.; Wan, H., Combining STRONG with screening designs for large-scale simulation optimization, IIE Transactions, 46, 4, 357-373, (2014)
[12] Chen, V. C.P.; Tsui, K.-L.; Barton, R. R.; Meckesheimer, M., A review on design, modeling, and applications of computer experiments, IIE Transactions, 38, 273-291, (2006)
[13] Chen, X.; Kim, K.-K., Building metamodels for quantile-based measures using sectioning, (Pasupathy, R.; Kim, S.-H.; Tolk, A.; Hill, R.; Kuhl, M. E., Proceedings of the 2013 winter simulation conference, (2013)), 521-532
[14] Chevalier, C.; Ginsbourger, D.; Bect, J.; Vazquez, E.; Picheny, V.; Richet, Y., Fast parallel Kriging-based stepwise uncertainty reduction with application to the identification of an excursion set, Technometrics, 56, 4, 455-465, (2014)
[15] Cousin, A.; Maatouk, H.; Rullière, D., Kriging of financialterm-structures, (2015)
[16] Crombecq, K.; Laermans, E.; Dhaene, T., Efficient space-filling and non-collapsing sequential design strategies for simulation-based modeling, European Journal of Operational Research, 214, 683-696, (2011)
[17] Dellino, G.; Kleijnen, J. P.C.; Meloni, C., Robust optimization in simulation: Taguchi and krige combined, INFORMS Journal on Computing, 24, 3, 471-484, (2012) · Zbl 06599283
[18] Efron, B., Frequentist accuracy of Bayesian estimates, Royal Statistical Society’s Series B Journal, 77, part 3, 617-646, (2015) · Zbl 1414.62089
[19] Fédou, J.-M.; Rendas, M.-J., Extending Morris method: identification of the interaction graph using cycle-equitable designs, Journal of Statistical Computation and Simulation, 85, 7, 1398-1419, (2015)
[20] Gordy, M. B.; Juneja, S., Nested simulation in portfolio risk measurement, Management Science, 56, 11, 1833-1848, (2010) · Zbl 1232.91622
[21] Helton, J. C.; Hansen, C. W.; Swift, P. N., Performance assessment for the proposed high-level radioactive waste repository at yucca mountain, nevada, Reliability Engineering and System Safety, 122, 1-6, (2014)
[22] Jalali, H.; Van Nieuwenhuyse, I., Simulation optimization in inventory replenishment: a classification, IIE Transactions, accepted, (2015)
[23] Jones, D. R.; Schonlau, M.; Welch, W. J., Efficient global optimization of expensive black-box functions, Journal of Global Optimization, 13, 455-492, (1998) · Zbl 0917.90270
[24] Kamiński, B., A method for updating of stochastic Kriging metamodels, European Journal of Operational Research, accepted, (2015) · Zbl 1346.62039
[25] Khuri, A. I.; Mukhopadhyay, S., Response surface methodology, Wiley Interdisciplinary Reviews: Computational Statistics, 2, 128-149, (2010)
[26] Kleijnen, J. P.C., An overview of the design and analysis of simulation experiments for sensitivity analysis, European Journal of Operational Research, 164, 2, 287-300, (2005) · Zbl 1068.90104
[27] Kleijnen, J. P.C., Kriging metamodeling in simulation: a review, European Journal of Operational Research, 192, 3, 707-716, (2009) · Zbl 1157.90544
[28] Kleijnen, J. P.C., Design and analysis of simulation experiments, (2015), Springer · Zbl 1321.62006
[29] Kleijnen, J. P.C.; Van Beers, W. C.M., Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations, Journal Operational Research Society, 64, 708-717, (2013)
[30] Kleijnen, J. P.C.; Beers, W. C.M. V.; Nieuwenhuyse, I. V., Constrained optimization in simulation: a novel approach, European Journal of Operational Research, 202, 1, 164-174, (2010) · Zbl 1189.90156
[31] Kleijnen, J. P.C.; Deflandre, D., Validation of regression metamodels in simulation: bootstrap approach, European Journal of Operational Research, 170, 1, 120-131, (2006) · Zbl 1330.62201
[32] Kleijnen, J. P.C.; Gaury, E. G.A., Short-term robustness of production-management systems: a case study, European Journal of Operational Research, 148, 2, 452-465, (2003) · Zbl 1036.90036
[33] Kleijnen, J. P.C.; Pierreval, H.; Zhang, J., Methodology for determining the acceptability of system designs in uncertain environments, European Journal of Operational Research, 209, 2, 176-183, (2011) · Zbl 1208.90088
[34] Koch, P.; Wagner, T.; Emmerich, M. T.M.; Bäck, T.; Konen, W., Efficient multi-criteria optimization on noisy machine learning problems, Applied Soft Computing, 29, 357-370, (2015)
[35] Law, A. M., Simulation modeling and analysis, (2015), McGraw-Hill, Boston
[36] Lee, K.-H.; Park, G.-J., A global robust optimization using Kriging based approximation model, Journal of the Japanese Society for Mechanical Engineering, 49, 3, 779-788, (2006)
[37] Loeppky, J. L.; Sacks, J.; Welch, W., Choosing the sample size of a computer experiment: a practical guide, Technometrics, 51, 4, 366-376, (2009)
[38] Lophaven, S. N.; Nielsen, H. B.; Sondergaard, J., DACE: A Matlab Kriging toolbox, version 2.0, (2002), IMM Technical University of Denmark, Kongens Lyngby
[39] Maatouk, H., & Bay, X. (2016). Gaussian process emulators for computer experiments with inequality constraints. arXiv:1606.01265v1. · Zbl 1371.65006
[40] Markiewicz, A.; Szczepańska, A., Optimal designs in multivariate linear models, Statistics & Probability Letters, 77, 426-430, (2007) · Zbl 1108.62074
[41] Montgomery, D. C., Design and analysis of experiments, (2009), Wiley, Hoboken, NJ
[42] Myers, R. H.; Montgomery, D. C.; Anderson-Cook, C. M., Response surface methodology: process and product optimization using designed experiments, (2009), Wiley, New York · Zbl 1269.62066
[43] Nelson, B. L., ‘some tactical problems in digital simulation’ for the next 10 years, Journal of Simulation, 10, 2-11, (2016)
[44] Preuss, M.; Wagner, T.; Ginsbourger, D., High-dimensional model-based optimization based on noisy evaluations of computer games, (Hamadi, Y.; Schoenauer, M., LION 6, LNCS 7219, (2012), Springer-Verlag Berlin Heidelberg), 145-159
[45] Qu, H.; Fu, M. C., Gradient extrapolated stochastic Kriging, ACM Transactions on Modeling and Computer Simulation, 24, 4, 23:1-23:25, (2014) · Zbl 1371.65011
[46] Qu, X., Statistical properties of rechtschaffner designs, Journal of Statistical Planning and Inference, 137, 2156-2164, (2007) · Zbl 1120.62058
[47] Rasmussen, C. E.; Williams, C., Gaussian processes for machine learning, (2006), MIT Press, Cambridge, Massachusetts · Zbl 1177.68165
[48] Ryan, K. J.; Bulutoglu, D. A., Minimum aberration fractional factorial designs with large n, Technometrics, 52, 2, 250-255, (2010)
[49] Saltelli, A.; Ratto, M.; Andres, T.; Campolongo, F.; Cariboni, J.; Gatelli, D., Global sensitivity analysis: The primer, (2008), Wiley · Zbl 1161.00304
[50] Santos, M. I.; Santos, P. M., Switching regression metamodels in stochastic simulation, European Journal of Operational Research, 251, 1, 142-147, (2016) · Zbl 1347.62135
[51] Shi, W.; Kleijnen, J. P.C.; Liu, Z., Factor screening for simulation with multiple responses: sequential bifurcation, European Journal of Operational Research, 237, 1, 136-147, (2014) · Zbl 1304.62109
[52] Shrivastava, A. K.; Ding, Y., Graph based isomorph-free generation of two-level regular fractional factorial designs, Journal of Statistical Planning and Inference, 140, 169-179, (2010) · Zbl 1178.62082
[53] Turner, A. J.; Balestrini-Robinson, S.; Mavris, D., Heuristics for the regression of stochastic simulations, Journal of Simulation, 7, 229-239, (2013)
[54] Ulaganathan, S.; Couckuyt, I.; Dhaene, T.; Laermans, E., On the use of gradients in Kriging surrogate models, (Tolk, A.; Diallo, S. Y.; Ryzhov, I. O.; Yilmaz, L.; Buckley, S.; Miller, J. A., Proceedings of the 2014 winter simulation conference, (2014)), 2692-2701
[55] Van Beers, W. C.M.; Kleijnen, J. P.C., Customized sequential designs for random simulation experiments: Kriging metamodeling and bootstrapping, European Journal of Operational Research, 186, 3, 1099-1113, (2008) · Zbl 1133.62060
[56] Wan, H.; Ankenman, B. E.; Nelson, B. L., Improving the efficiency and efficacy of controlled sequential bifurcation for simulation factor screening, INFORMS Journal on Computing, 22, 3, 482-492, (2010) · Zbl 1243.62110
[57] Wei, P.; Lu, Z.; Song, J., Variable importance analysis: a comprehensive review, Reliability Engineering and System Safety, 142, 399-432, (2015)
[58] Woods, D. C., & Lewis, S. M. (2015). Design of experiments for screening. arXiv:1510.05248.
[59] Wu, C. F.J.; Hamada, M., Experiments: planning, analysis, and parameter design optimization, (2009), Wiley, New York
[60] Yanikoğlu, I.; den Hertog, D.; Kleijnen, J. P.C., Adjustable robust parameter design with unknown distributions, IIE Transactions, 48, 3, 298-312, (2016)
[61] Yin, J.; Ng, S. H.; Ng, K. M., A study on the effects of parameter estimation on Kriging model’s prediction error in stochastic simulation, (Rossini, M. D.; Hill, R. R.; Johansson, B.; Dunkin, A.; Ingalls, R. G., Proceedings of the 2009 winter simulation conference, (2009)), 674-685
[62] Yuan, J.; Ng, S. H., Calibration, validation and prediction in random simulation models: gaussian process metamodels and a Bayesian integrated solution, Transactions on Modeling and Computer Simulation, 25, 3, 1-25, (2015) · Zbl 1371.65016
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.