# zbMATH — the first resource for mathematics

Designing combined physical and computer experiments to maximize prediction accuracy. (English) Zbl 06917618
Summary: Combined designs for experiments involving a physical system and a simulator of the physical system are evaluated in terms of their accuracy of predicting the mean of the physical system. Comparisons are made among designs that are (1) locally optimal under the minimum integrated mean squared prediction error criterion for the combined physical system and simulator experiments, (2) locally optimal for the physical or simulator experiments, with a fixed design for the component not being optimized, (3) maximin augmented nested Latin hypercube, and (4) I-optimal for the physical system experiment and maximin Latin hypercube for the simulator experiment. Computational methods are proposed for constructing the designs of interest. For a large test bed of examples, the empirical mean squared prediction errors are compared at a grid of inputs for each test surface using a statistically calibrated Bayesian predictor based on the data from each design. The prediction errors are also studied for a test bed that varies only the calibration parameter of the test surface. Design recommendations are given.

##### MSC:
 62 Statistics
Matlab; SAS
Full Text:
##### References:
 [1] Audze, P.; Eglais, V., New approach for planning out of experiments, Probl Dynam. Strengths, 35, 104-107, (1977) [2] Ba, S.; Joseph, V. R., Composite Gaussian process models for emulating expensive functions, Ann. Appl. Stat., 6, 4, 1838-1860, (2012) · Zbl 1257.62089 [3] Crary, S., Design of computer experiments for metamodel generation, Analog Integr. Circuits Signal Process., 32, (2002) [4] Crary, S., Stormann, J., 2015. Four-Point, 2D, Free-Ranging, IMSPE-Optimal, Twin-Point Designs. ArXiv e-prints. [5] Fang, K.; Wang, Y., Number theoretic methods in statistics, (1994), Chapman & Hall · Zbl 0925.65263 [6] Gattiker, J. R., Gaussian process models for simulation analysis (GPM/SA) command, function, and data structure reference. tech. rep. LA-UR-08-08057, (2008), Los Alamos National Laboratory [7] Hardin, R. H.; Sloane, N. J.A., A new approach to the construction of optimal designs, J. Statist. Plann. Inference, 37, 339-369, (1993) · Zbl 0799.62082 [8] Higdon, D.; Gattiker, J.; Williams, B.; Rightley, M., Computer model calibration using high dimensional output, J. Amer. Statist. Assoc., 103, 570-583, (2008) · Zbl 05564511 [9] Higdon, D.; Kennedy, M.; Cavendish, J.; Cafeo, J.; Ryne, R., Combining field data and computer simulations for calibration and prediction, SIAM J. Sci. Comput., 26, 448-466, (2004) · Zbl 1072.62018 [10] JMP, 1989-2007. Version 11. SAS Institute Inc., Cary, NC. [11] Johnson, M. E.; Moore, L. M.; Ylvisaker, D., Minimax and maximin distance designs, J. Statist. Plann. Inference, 26, 131-148, (1990) [12] Kennedy, J., Eberhart, R., 1995. Particle swarm optimization. In: IEEE International Conference on Neural Networks, 1995. Proceedings. Vol. 4. pp. 1942-1948. [13] Kennedy, M. C.; O’Hagan, A., Bayesian calibration of computer models (with discussion), J. R. Stat. Soc. Ser. B Stat. Methodol., 63, 425-464, (2001) · Zbl 1007.62021 [14] Leatherman, E. R.; Dean, A. M.; Santner, T. J., Computer experiment designs via particle swarm optimization, (Melas, V.; Mignani, S.; Monari, P.; Salmaso, L., Topics in Statistical Simulation: Research Papers from the 7th International Workshop on Statistical Simulation. Vol. 114, (2014), Springer), 309-317 [15] Leatherman, E. R.; Guo, H.; Gilbert, S. L.; Hutchinson, I.; Maher, S. A.; Santner, T. J., Using a statistically calibrated biphasic finite element model of the human knee joint to identify robust designs for a mensical substitute, J. Biomech. Eng., 136, 7, (2014) [16] Leatherman, E. R.; Santner, T. J.; Dean, A. M., Computer experiment designs for prediction. tech. rep. 895, (2016), The Ohio State University, Department of Statistics [17] Liefvendahl, M.; Stocki, R., A study on algorithms for optimization of Latin hypercubes, J. Statist. Plann. Inference, 136, 3231-3247, (2006) · Zbl 1094.62084 [18] Loeppky, J.L., Williams, B.J., Welch, W.J., 2010. A critical assessment of computer model calibration (unpublished manuscript). [19] MATLAB. 8.5.0.197613 (R2015a). The MathWorks Inc., Natick, Massachusetts, 2015. [20] Moon, H.; Santner, T.; Dean, A. M., Algorithms for generating maximin Latin hypercube and orthogonal designs, J. Stat. Theory Pract., 5, 81-98, (2011) · Zbl 05902637 [21] Niederreiter, H., Quasi-Monte Carlo methods and pseudo-random numbers, Bull. Amer. Math. Soc., 84, 957-1041, (1978) · Zbl 0404.65003 [22] Niederreiter, H., Random number generation and quasi-Monte Carlo methods, (1992), SIAM Philadelphia · Zbl 0761.65002 [23] Owen, A. B., Randomly permuted $$(t, m, s)$$-nets and $$(t, s)$$ sequences, (Niederreiter, H.; Shiue, P. J.-S., Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, (1995), Springer-Verlag New York), 299-317 · Zbl 0831.65024 [24] Ranjan, P.; Lu, W.; Bingham, D.; Reese, S.; Williams, B. J.; Chou, C.-C.; Doss, F.; Grosskopf, M.; Holloway, J. P., Follow-up experimental designs for computer models and physical processes, J. Stat. Theory Pract., 5, 1, 119-136, (2011) [25] Santner, T. J.; Williams, B. J.; Notz, W. I., The design and analysis of computer experiments, (2003), Springer Verlag New York · Zbl 1041.62068 [26] Sobol´, I.; Levitan, Y., On the use of variance reducing multipliers in Monte Carlo computations of a global sensitivity index, Comput. Phys. Comm., 117, 1-2, 52-61, (1999) · Zbl 1018.65002 [27] Studden, W. J., Optimal designs for integrated variance in polynomial regression, (Gupta, S. S.; Moore, D. S., Statistical Decision Theory and Related Topics. Vol. II, (1977), Academic Press New York), 411-420 · Zbl 0422.62071 [28] Trosset, M. W., (The krigifier: a procedure for generating pseudorandom nonlinear objective functions for computational experimentation. Tech. Rep. 35, (1999), Institute for Computer Applications in Science and Engineering, NASA Langley Research Center) [29] van Dam, E., den Hertog, D., Husslage, B., Rennen, G., 2013. Space-filling designs. http://www.spacefillingdesigns.nl/, (accessed: October-December 2015). · Zbl 1284.90100 [30] Welch, W. J., Aced: algorithms for the construction of experimental designs, Amer. Statist., 39, 146, (1985) [31] Williams, B. J.; Loeppky, J. L.; Moore, L. M.; Macklem, M. S., Batch sequential design to achieve predictive maturity with calibrated computer models, Reliab. Eng. Syst. Saf., 96, 9, 1208-1219, (2011) [32] Xiong, Y.; Chen, W.; Apley, D.; Ding, X., A non-stationary covariance-based Kriging method for metamodelling in engineering design, Internat. J. Numer. Methods Engrg., 71, 6, 733-756, (2007) · Zbl 1194.74553 [33] Yang, X.-S., Engineering optimization: an introduction with metaheuristic applications, (2010), Wiley Publishing
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.