×

zbMATH — the first resource for mathematics

Surrogate-based methods for black-box optimization. (English) Zbl 1366.90196
Summary: In this paper, we survey methods that are currently used in black-box optimization, that is, the kind of problems whose objective functions are very expensive to evaluate and no analytical or derivative information is available. We concentrate on a particular family of methods, in which surrogate (or meta) models are iteratively constructed and used to search for global solutions.

MSC:
90C30 Nonlinear programming
90C15 Stochastic programming
90C59 Approximation methods and heuristics in mathematical programming
PDF BibTeX Cite
Full Text: DOI
References:
[1] Abramson , M.A. Audet , C. Couture , G. Dennis , J.E. Le Digabel , S. Tribes , C. The NOMAD project https://www.gerad.ca/nomad/
[2] Akhtar, Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection, Journal of Global Optimization 64 pp 17– (2016) · Zbl 1339.90293
[3] Audet, Mathematics Without Boundaries pp 31– (2014)
[4] Audet, A pattern search filter method for nonlinear programming without derivatives, SIAM Journal on Optimization 14 (4) pp 980– (2004) · Zbl 1073.90066
[5] Audet, Mesh adaptive direct search algorithms for constrained optimization, SIAM Journal on Optimization 17 pp 188– (2006) · Zbl 1112.90078
[6] Audet, A progressive barrier for derivative-free nonlinear programming, SIAM Journal on Optimization 20 (1) pp 445– (2009) · Zbl 1187.90266
[7] Audze, New approach for planning out of experiments, Problems of Dynamics and Strengths 35 pp 104– (1977)
[8] Bates , S. Sienz , J. Toropov , V.V. 2004 Formulation of the optimal Latin hypercube design of experiments using a permutation genetic algorithm 5th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference Palm Springs CA 1 7
[9] Beicip-Franlab, Pumaflow Reservoir Simulator Reference Manual (2012)
[10] Björkman, Global optimization of costly nonconvex functions using radial basis functions, Optimization and Engineering 1 pp 373– (2000) · Zbl 1035.90061
[11] Booker, A rigorous framework for optimization of expensive functions by surrogates, Structural Optimization 17 (1) pp 1– (1999)
[12] Bouzarkouna , Z. Ding , D.Y. Auger , A. 2010 Using evolution strategy with meta-models for well placement optimization Proceedings of the XII European Conference on the Mathematics of Oil Recovery
[13] Cassioli, Global optimization of expensive black box problems with a known lower bound, Journal of Global Optimization 57 pp 177– (2013) · Zbl 1277.90098
[14] Conn, Use of quadratic models with mesh-adaptive direct search for constrained black box optimization, Optimization Methods and Software 28 (1) pp 139– (2013) · Zbl 1270.90073
[15] Conn , A.R. Scheinberg , K. Vicente , L.N. 2009 Introduction to Derivative-Free Optimization MPS/SIAM Series on Optimization SIAM Philadelphia, PA · Zbl 1163.49001
[16] Costa , A. Nannicini , G. 2014 RBFOpt: an open-source library for black-box optimization with costly function evaluations Optimization Online
[17] Cox , D.D. John , S. 1992 A statistical method for global optimization Proceedings of the IEEE International Conference on Systems, Man and Cybernetics 1241 1246
[18] Cox, Multidisciplinary Design Optimization: State of the Art pp 315– (1997)
[19] Fang, Centered 2-discrepancy of random sampling and Latin hypercube design, and construction of uniform designs, Mathematics of Computation 71 pp 275– (2002) · Zbl 0977.68091
[20] Fialho , A. Hamadi , Y. Schoenauer , M. 2012 A multi-objective approach to balance buildings construction cost and energy efficiency ECAI’12 961 966
[21] Forrester, Engineering Design via Surrogate Modelling, a Practical Guide (2008)
[22] Gasca, Polynomial interpolation in several variables, Advances in Computational Mathematics 12 pp 377– (2000) · Zbl 0943.41001
[23] Grosso, Finding maximin Latin hypercube designs by iterated local search heuristics, European Journal of Operational Research 197 pp 541– (2009) · Zbl 1159.90523
[24] Gutmann, A radial basis function method for global optimization, Journal of Global Optimization 19 pp 201– (2001) · Zbl 0972.90055
[25] Holmström, An adaptive radial basis algorithm (ARBF) for expensive black-box global optimization, Journal of Global Optimization 41 (3) pp 447– (2007) · Zbl 1152.90609
[26] Hussain, Metamodeling: radial basis functions versus polynomials, European Journal of Operations Research 138 pp 142– (2002) · Zbl 1009.90135
[27] Husslage, Space-filling Latin hypercube designs for computer experiments, Optimization and Engineering 12 pp 611– (2011) · Zbl 1284.90100
[28] Jakobsson, A method for simulation based optimization using radial basis functions, Optimization and Engineering 11 pp 501– (2010) · Zbl 1243.65068
[29] Ji , Y. Kim , S. Lu , W.X. 2013 A new framework for combining global and local methods in black box optimization Optimization Online
[30] Johnson, Minimax and maximin distance designs, Journal of Statistical Planning and Inference 26 pp 131– (1990)
[31] Jones, A taxonomy of global optimization methods based on response surfaces, Journal of Global Optimization 21 pp 345– (2001) · Zbl 1172.90492
[32] Jones, Efficient global optimization of expensive black-box functions, Journal of Global Optimization 13 pp 445– (1998) · Zbl 0917.90270
[33] Koehler, Handbook of Statistics, 13: Design and Analysis of Experiments pp 261– (1996)
[34] Kushner, A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise, Journal of Basic Engineering 86 pp 97– (1964)
[35] Digabel, Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm, ACM Transactions on Mathematical Software 37 (4) pp 1– (2011) · Zbl 1365.65172
[36] Lizon , C. D’Ambrosio , C. Liberti , L. Le Ravalec , M. Sinoquet , D. 2014 A mixed-integer nonlinear optimization approach for well placement and geometry Proceedings of the XIV European Conference on the Mathematics of Oil Recovery
[37] McKay, A comparison of three methods for selecting values of input variables in the analysis of output from a computer code, Technometrics 21 pp 239– (1979) · Zbl 0415.62011
[38] Mockus, Application of Bayesian approach to numerical methods of global and stochastic optimization, Journal of Global Optimization 4 pp 347– (1994) · Zbl 0801.90099
[39] Morris, Exploratory designs for computational experiment, Journal of Statistical Planning and Inference 43 pp 381– (1995) · Zbl 0813.62065
[40] Mueller , J. 2014 MATSuMoTo: The MATLAB surrogate model toolbox for computationally expensive black-box global optimization problems arXiv:1404.4261
[41] Müller, Mixture surrogate models based on Dempster-Shafer theory for global optimization problems, Journal of Global Optimization 51 (1) pp 79– (2011) · Zbl 1230.90155
[42] Müller, Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems, Journal of Global Optimization 60 (2) pp 123– (2014) · Zbl 1312.90064
[43] Müller, SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems, Computers & Operations Research 40 (5) pp 1383– (2013) · Zbl 1352.90067
[44] Müller, SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications, Journal of Global Optimization 59 (4) pp 865– (2014) · Zbl 1305.90305
[45] Nakayama , H. Arakawa , M. Washino , K. 2003 Using support vector machines in optimization for black-box objective functions Proceedings of the International Joint Conference on Neural Networks, 2003 1617 1622
[46] Nelder, A simplex method for function minimization, Computer Journal 7 pp 308– (1965) · Zbl 0229.65053
[47] Onwunalu, Application of a particle swarm optimization algorithm for determining optimum well location and type, Computational Geosciences 14 pp 183– (2010) · Zbl 1185.86024
[48] Regis, Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions, Computers and Operations Research 38 pp 837– (2011) · Zbl 1434.90109
[49] Regis, Constrained global optimization of expensive black-box functions using radial basis functions, Journal of Global Optimization 31 pp 153– (2005) · Zbl 1274.90511
[50] Regis, Improved strategies for radial basis function methods for global optimization, Journal of Global Optimization 37 pp 113– (2007a) · Zbl 1149.90120
[51] Regis, Parallel radial basis function methods for the global optimization of expensive functions, European Journal of Operational Research 182 pp 514– (2007b) · Zbl 1178.90279
[52] Regis, A stochastic radial basis function method for the global optimization of expensive functions, INFORMS Journal on Computing 19 pp 497– (2007c) · Zbl 1241.90192
[53] Regis, A quasi-multistart framework for global optimization of expensive functions using response surface models, Journal of Global Optimization 56 pp 1719– (2013) · Zbl 1275.90068
[54] Sacks, Design and analysis of computer experiments, Statistical Science 4 pp 409– (1989) · Zbl 0955.62619
[55] Santner, The Design and Analysis of Computer Experiments (2003) · Zbl 1041.62068
[56] Shewry, Maximum entropy sampling, Journal of Applied Statistics 14 pp 165– (1987)
[57] Sóbester, Engineering design applications of surrogate-assisted optimization techniques, Optimization and Engineering 15 (1) pp 243– (2012) · Zbl 1308.65097
[58] Stinstra, Constrained maximin designs for computer experiments, Technometrics 45 pp 340– (2003)
[59] Stocki, A method to improve design reliability using Optimal Latin hypercube sampling, Computer Assisted Mechanics and Engineering Sciences 12 pp 393– (2005)
[60] Talgorn, Statistical surrogate formulations for simulation-based design optimization, Journal of Mechanical Design 137 (2) pp 1– (2015)
[61] Torczon, On the convergence of pattern search algorithms, SIAM Journal on Optimization 7 pp 1– (1997) · Zbl 0884.65053
[62] Vazquez, Convergence properties of the expected improvement algorithm with fixed mean and covariance functions, Journal of Statistical Planning and Inference 140 pp 3088– (2010) · Zbl 1419.62200
[63] Viana, Multiple surrogates: how cross-validation errors can help us to obtain the best predictor, Structural and Multidisciplinary Optimization 39 (4) pp 439– (2009) · Zbl 06227712
[64] Viana, Efficient global optimization algorithm assisted by multiple surrogate techniques, Journal of Global Optimization 56 (2) pp 669– (2013) · Zbl 1275.90072
[65] Ye, Algorithmic construction of optimal symmetric Latin hypercube designs, Journal of Statistical Planning and Inference 90 pp 145– (2000) · Zbl 1109.62329
[66] Žilinskas, A review of statistical models for global optimization, Journal of Global Optimization 2 pp 145– (1992) · Zbl 0768.90071
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.