Sequential design of computer experiments for the estimation of a probability of failure. (English) Zbl 1252.62081

Summary: This paper deals with the problem of estimating the volume of the excursion set of a function \(f:\mathbb{R}^d \to \mathbb{R}\) above a given threshold, under a probability measure on \(\mathbb{R}^d\) that is assumed to be known. In the industrial world, this corresponds to the problem of estimating a probability of failure of a system. When only an expensive-to-simulate model of the system is available, the budget for simulations is usually severely limited and therefore classical Monte Carlo methods ought to be avoided. One of the main contributions of this article is to derive SUR (stepwise uncertainty reduction) strategies from a Bayesian formulation of the problem of estimating a probability of failure. These sequential strategies use a Gaussian process model of f and aim at performing evaluations of f as efficiently as possible to infer the value of the probability of failure. We compare these strategies to other strategies also based on a Gaussian process model for estimating a probability of failure.


62L05 Sequential statistical design
62M09 Non-Markovian processes: estimation
62N05 Reliability and life testing
62F15 Bayesian inference
65C05 Monte Carlo methods


KrigInv; EGO
Full Text: DOI arXiv


[1] Arnaud, A., Bect, J., Couplet, M., Pasanisi, A., Vazquez, E.: Évaluation d’un risque d’inondation fluviale par planification séquentielle d’expériences. In: 42èmes Journées de Statistique (2010)
[2] Au, S.K., Beck, J.: Estimation of small failure probabilities in high dimensions by subset simulation. Probab. Eng. Mech. 16(4), 263–277 (2001)
[3] Bayarri, M.J., Berger, J.O., Paulo, R., Sacks, J., Cafeo, J.A., Cavendish, J., Lin, C.H., Tu, J.: A framework for validation of computer models. Technometrics 49(2), 138–154 (2007)
[4] Berry, D.A., Fristedt, B.: Bandit Problems: Sequential Allocation of Experiments. Chapman & Hall, London (1985) · Zbl 0659.62086
[5] Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. 1. Athena Scientific, Nashua (1995)
[6] Bichon, B.J., Eldred, M.S., Swiler, L.P., Mahadevan, S., McFarland, J.M.: Efficient global reliability analysis for nonlinear implicit performance functions. AIAA J. 46(10), 2459–2468 (2008)
[7] Bjerager, P.: On computational methods for structural reliability analysis. Struct. Saf. 9, 76–96 (1990)
[8] Borri, A., Speranzini, E.: Structural reliability analysis using a standard deterministic finite element code. Struct. Saf. 19(4), 361–382 (1997)
[9] Box, G.E.P., Draper, N.R.: Empirical Model-Building and Response Surfaces. Wiley, New York (1987) · Zbl 0614.62104
[10] Bucher, C.G., Bourgund, U.: A fast and efficient response surface approach for structural reliability problems. Struct. Saf. 7(1), 57–66 (1990)
[11] Chilès, J.P., Delfiner, P.: Geostatistics: Modeling Spatial Uncertainty. Wiley, New York (1999) · Zbl 0922.62098
[12] Currin, C., Mitchell, T., Morris, M., Ylvisaker, D.: Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. J. Am. Stat. Assoc. 86(416), 953–963 (1991)
[13] Deheeger, F.: Couplage mécano-fiabiliste : 2SMART–méthodologie d’apprentissage stochastique en fiabilité. Ph.D. thesis, Université Blaise Pascal, Clermont II (2008)
[14] Deheeger, F., Lemaire, M.: Support vector machine for efficient subset simulations: 2SMART method. In: Kanda, J., Takada, T., Furuta, H. (eds.) 10th International Conference on Application of Statistics and Probability in Civil Engineering, Proceedings and Monographs in Engineering, Water and Earth Sciences, pp. 259–260. Taylor & Francis, London (2007)
[15] Echard, B., Gayton, N., Lemaire, M.: Kriging-based Monte Carlo simulation to compute the probability of failure efficiently: AK-MCS method. In: 6èmes Journées Nationales de Fiabilité, Toulouse, France, 24–26 mars 2010a
[16] Echard, B., Gayton, N., Lemaire, M.: Structural reliability assessment using kriging metamodel and active learning. In: IFIP WG 7.5 Working Conference on Reliability and Optimization of Structural Systems (2010b)
[17] Fleuret, F., Geman, D.: Graded learning for object detection. In: Proceedings of the Workshop on Statistical and Computational Theories of Vision of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR/SCTV) (1999) · Zbl 1012.68713
[18] Frazier, P.I., Powell, W.B., Dayanik, S.: A knowledge-gradient policy for sequential information collection. SIAM J. Control Optim. 47(5), 2410–2439 (2008) · Zbl 1274.62155
[19] Ginsbourger, D.: Métamodèles multiples pour l’approximation et l’optimisation de fonctions numériques multivariables. Ph.D. thesis, Ecole nationale supérieure des Mines de Saint-Etienne (2009)
[20] Ginsbourger, D., Le Riche, R., Carraro, L.: Kriging is well-suited to parallelize optimization. In: Hiot, L.M., Ong, Y.S., Tenne, Y., Goh, C.K. (eds.) Computational Intelligence in Expensive Optimization Problems. Adaptation Learning and Optimization, vol. 2, pp. 131–162. Springer, Berlin (2010)
[21] Handcock, M.S., Stein, M.L.: A Bayesian analysis of kriging. Technometrics 35(4), 403–410 (1993)
[22] Hornik, K.: The R FAQ (2010). http://CRAN.R-project.org/doc/FAQ/R-FAQ.html . ISBN 3-900051-08-9
[23] Hurtado, J.E.: An examination of methods for approximating implicit limit state functions from the viewpoint of statistical learning theory. Struct. Saf. 26(3), 271–293 (2004)
[24] Hurtado, J.E.: Filtered importance sampling with support vector margin: a powerful method for structural reliability analysis. Struct. Saf. 29(1), 2–15 (2007)
[25] Jones, D.R., Schonlau, M., William, J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998) · Zbl 0917.90270
[26] Kennedy, M., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc., Ser. B, Stat. Methodol. 63(3), 425–464 (2001) · Zbl 1007.62021
[27] Kimeldorf, G.S., Wahba, G.: A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann. Math. Stat. 41(2), 495–502 (1970) · Zbl 0193.45201
[28] Kushner, H.J.: A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. J. Basic Eng. 86, 97–106 (1964)
[29] Loeppky, J.L., Sacks, J., Welch, W.J.: Choosing the sample size of a computer experiment: a practical guide. Technometrics 51(4), 366–376 (2009)
[30] Mockus, J.: Bayesian Approach to Global Optimization. Theory and Applications. Kluwer Academic, Dordrecht (1989) · Zbl 0693.49001
[31] Mockus, J., Tiesis, V., Zilinskas, A.: The application of Bayesian methods for seeking the extremum. In: Dixon, L., Szego, E.G. (eds.) Towards Global Optimization, vol. 2, pp. 117–129. Elsevier, Amsterdam (1978) · Zbl 0394.90090
[32] Oakley, J.: Estimating percentiles of uncertain computer code outputs. J. R. Stat. Soc., Ser. C 53(1), 83–93 (2004) · Zbl 1111.62395
[33] Oakley, J., O’Hagan, A.: Bayesian inference for the uncertainty distribution of computer model outputs. Biometrika 89(4) (2002)
[34] Oakley, J., O’Hagan, A.: Probabilistic sensitivity analysis of complex models: a Bayesian approach. J. R. Stat. Soc., Ser. B, Stat. Methodol. 66(3), 751–769 (2004) · Zbl 1046.62027
[35] O’Hagan, A.: Curve fitting and optimal design for prediction. J. R. Stat. Soc., Ser. B, Methodol. 40(1), 1–42 (1978)
[36] Parzen, E.: An approach to time series analysis. Ann. Math. Stat. 32, 951–989 (1962) · Zbl 0107.13801
[37] Paulo, R.: Default priors for Gaussian processes. Ann. Stat. 33(2), 556–582 (2005) · Zbl 1069.62030
[38] Picheny, V., Ginsbourger, D.: KrigInv: Kriging-based inversion for deterministic and noisy computer experiments, version 1.1 (2011). http://cran.r-project.org/web/packages/KrigInv
[39] Picheny, V., Ginsbourger, D., Roustant, O., Haftka, R.T., Kim, N.H.: Adaptive designs of experiments for accurate approximation of target regions. J. Mech. Des. 132(7), 1–9 (2010)
[40] Piera-Martinez, M.: Modélisation des comportements extrêmes en ingénierie. Ph.D. thesis, Université Paris Sud, Paris XI (2008)
[41] Pradlwarter, H., Schuëller, G., Koutsourelakis, P., Charmpis, D.: Application of line sampling simulation method to reliability benchmark problems. Struct. Saf. 29(3), 208–221 (2007). A Benchmark Study on Reliability in High Dimensions
[42] Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C. The Art of Scientific Computing, 2nd edn. Cambridge University Press, Cambridge (1992) · Zbl 0845.65001
[43] Rajashekhar, M.R., Ellingwood, B.R.: A new look at the response surface approach for reliability analysis. Struct. Saf. 12(3), 205–220 (1993)
[44] Ranjan, P., Bingham, D., Michailidis, G.: Sequential experiment design for contour estimation from complex computer codes. Technometrics 50(4), 527–541 (2008)
[45] Rubinstein, R., Kroese, D.: The Cross-Entropy Method. Springer, Berlin (2004) · Zbl 1061.90032
[46] Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P.: Design and analysis of computer experiments. Stat. Sci. 4(4), 409–435 (1989) · Zbl 0955.62619
[47] Santner, T.J., Williams, B.J., Notz, W.: The Design and Analysis of Computer Experiments. Springer, Berlin (2003) · Zbl 1041.62068
[48] Schueremans, L.: Probabilistic evaluation of structural unreinforced masonry. Ph.D. thesis, Catholic University of Leuven (2001)
[49] Schueremans, L., Gemert, D.V.: Benefit of splines and neural networks in simulation based structural reliability analysis. Struct. Saf. 27(3), 246–261 (2005)
[50] Stein, M.L.: Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York (1999) · Zbl 0924.62100
[51] Vazquez, E., Bect, J.: A sequential Bayesian algorithm to estimate a probability of failure. In: Proceedings of the 15th IFAC Symposium on System Identification, SYSID 2009, Saint-Malo, France (2009)
[52] Vazquez, E., Piera-Martinez, M.: Estimation du volume des ensembles d’excursion d’un processus gaussien par krigeage intrinsèque. In: 39ème Journées de Statistiques Conférence Journée de Statistiques, Angers, France (2007)
[53] Vestrup, E.M.: The Theory of Measures and Integration. Wiley, New York (2003) · Zbl 1059.28001
[54] Villemonteix, J.: Optimisation de fonctions coûteuses. Ph.D. thesis, Université Paris-Sud XI, Faculté des Sciences d’Orsay (2008)
[55] Villemonteix, J., Vazquez, E., Walter, E.: An informational approach to the global optimization of expensive-to-evaluate functions. J. Glob. Optim. 44(4), 509–534 (2009) · Zbl 1180.90253
[56] Waarts, P.H.: Structural reliability using finite element methods: an appraisal of DARS. Ph.D. thesis, Delft University of Technology (2000)
[57] Welch, W.J., Buck, R.J., Sacks, J., Wynn, H.P., Mitchell, T.J., Morris, M.D.: Screening, predicting and computer experiments. Technometrics 34, 15–25 (1992)
[58] Yaglom, A.M.: Correlation Theory of Stationary and Related Random Functions I: Basic Results. Springer Series in Statistics. Springer, New York (1986)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.