zbMATH — the first resource for mathematics

Optimization of expensive black-box problems via gradient-enhanced Kriging. (English) Zbl 1439.90078
Summary: This paper explores the use of Gradient-enhanced Kriging for optimization of expensive black-box design problems, which is not completely limited by the conventional Efficient Global Optimization algorithm framework. Specifically, we give the best linear unbiased predictor and mean squared prediction error of the partial derivatives of Gradient-enhanced Kriging and then propose a measure named “Approximate Probability of Stationary Point” to estimate the approximate probability of a candidate infill point be a stationary point of the underlying function. When it comes to the selection of infill point, we not only maximize the well-known Expected Improvement but also evaluate the Approximate Probability of Stationary Point as a “double-check” step. Then the infill decision is made according to the extent of consistency between these two quantities. Furthermore, to examine whether the optimization process will gain from sparing more costs for response evaluation, we investigate also the cases that the gradient evaluation step is conditionally skipped in some iterations. Three new infill criteria are proposed and experimented with three analytical test functions and an airfoil optimal shape design. Results show that the optimization performance can be improved by exploiting the auxiliary gradient information in the proposed way.
Reviewer: Reviewer (Berlin)

90C56 Derivative-free methods and methods using generalized derivatives
60G25 Prediction theory (aspects of stochastic processes)
62K05 Optimal statistical designs
90C26 Nonconvex programming, global optimization
Full Text: DOI
[1] Forrester, A. I.J.; Keane, A. J., Recent advances in surrogate-based optimization, Prog. Aerosp. Sci., 45, 50-79 (2009)
[2] Viana, F. A.C.; Simpson, T. W.; Balabanov, V.; Toropov, V., Special section on multidisciplinary design optimization: Metamodeling in multidisciplinary design optimization: How far have we really come?, AIAA J., 52, 670-690 (2014)
[3] Vu, K. K.; D’Ambrosio, C.; Hamadi, Y.; Liberti, L., Surrogate-based methods for black-box optimization, Int. Trans. Oper. Res., 24 (2017) · Zbl 1366.90196
[4] Martins, J.; Kennedy, G., Enabling large-scale multidisciplinary design optimization through adjoint sensitivity analysis, (AIAA Scitech 2019 Forum (2019), American Institute of Aeronautics and Astronautics)
[5] Jameson, A., Aerodynamic design via control theory, J. Sci. Comput., 3, 233-260 (1988) · Zbl 0676.76055
[6] Giles, M. B.; Pierce, N. A., An introduction to the adjoint approach to design, Flow Turbul. Combust., 65, 393-415 (2000) · Zbl 0996.76023
[7] Giannakoglou, K. C.; Papadimitriou, D. I.; Kampolis, I. C., Aerodynamic shape design using evolutionary algorithms and new gradient-assisted metamodels, Comput. Methods Appl. Mech. Engrg., 195, 6312-6329 (2006) · Zbl 1178.76310
[8] Laurent, L.; Le Riche, R.; Soulier, B.; Boucard, P.-A., An overview of gradient-enhanced metamodels with applications, Arch. Comput. Methods Eng., 26, 61-106 (2019)
[9] Morris, M. D.; Mitchell, T. J.; Ylvisaker, D., Bayesian design and analysis of computer experiments: Use of derivatives in surface prediction, Technometrics, 35, 243-255 (1993) · Zbl 0785.62025
[10] Leary, S. J.; Bhaskar, A.; Keane, A. J., Global approximation and optimization using adjoint computational fluid dynamics codes, AIAA J., 42, 631-641 (2004)
[11] Laurenceau, J.; Meaux, M.; Montagnac, M.; Sagaut, P., Comparison of gradient-based and gradient-enhanced response-surface-based optimizers, AIAA J., 48, 981-994 (2010)
[12] March, A.; Willcox, K.; Wang, Q., Gradient-based multifidelity optimisation for aircraft design using Bayesian model calibration, Aeronaut. J., 115, 729-738 (2011)
[13] Backhaus, J.; Aulich, M.; Frey, C.; Lengyel, T.; Voß, C., Gradient enhanced surrogate models based on adjoint CFD methods for the design of a counter rotating turbofan, (ASME Turbo Expo 2012: Turbine Technical Conference and Exposition (2012), The American Society of Mechanical Engineers), 2319-2329
[14] Yamazaki, W.; Mavriplis, D. J., Derivative-enhanced variable fidelity surrogate modeling for aerodynamic functions, AIAA J., 51, 126-137 (2013)
[15] Han, Z.-H.; Zhang, Y.; Song, C.-X.; Zhang, K.-S., Weighted gradient-enhanced kriging for high-dimensional surrogate modeling and design optimization, AIAA J., 55, 4330-4346 (2017)
[16] Li, Z.; Zheng, X., Review of design optimization methods for turbomachinery aerodynamics, Prog. Aerosp. Sci., 93, 1-23 (2017)
[17] Chen, L.; Qiu, H.; Gao, L.; Jiang, C.; Yang, Z., A screening-based gradient-enhanced Kriging modeling method for high-dimensional problems, Appl. Math. Model., 69, 15-31 (2019)
[18] Leary, S. J.; Bhaskar, A.; Keane, A. J., A derivative based surrogate model for approximating and optimizing the output of an expensive computer simulation, J. Global Optim., 30, 39-58 (2004) · Zbl 1136.90302
[19] Laurent, L.; Boucard, P.-A.; Soulier, B., Generation of a cokriging metamodel using a multiparametric strategy, Comput. Mech., 51, 151-169 (2013) · Zbl 1312.65102
[20] Laurent, L.; Boucard, P. A.; Soulier, B., A dedicated multiparametric strategy for the fast construction of a cokriging metamodel, Comput. Struct., 124, 61-73 (2013)
[21] Liem, R. P.; Mader, C. A.; Martins, J. R.R. A., Surrogate models and mixtures of experts in aerodynamic performance prediction for aircraft mission analysis, Aerosp. Sci. Technol., 43, 126-151 (2015)
[22] Lockwood, B. A.; Anitescu, M., Gradient-enhanced universal kriging for uncertainty propagation, Nucl. Sci. Eng., 170, 168-195 (2012)
[23] Solak, E.; Murray-Smith, R.; Leithead, W. E.; Leith, D. J.; Rasmussen, C. E., Derivative observations in Gaussian process models of dynamic systems, Adv. Neural Inf. Process. Syst., 1057-1064 (2003)
[24] Alexandrov, N. M.; Dennis, J. E.; Lewis, R. M.; Torczon, V., A trust-region framework for managing the use of approximation models in optimization, Struct. Optim., 15, 16-23 (1998)
[25] Krige, D. G., A statistical approach to some basic mine valuation problems on the Witwatersrand, J. South. Afr. Inst. Min. Metall., 52, 119-139 (1951)
[26] Matheron, G., Principles of geostatistics, Econ. Geol., 58, 1246-1266 (1963)
[27] Sacks, J.; Welch, W. J.; Mitchell, T. J.; Wynn, H. P., Design and analysis of computer experiments, Statist. Sci., 4, 409-423 (1989) · Zbl 0955.62619
[28] Williams, C. K.; Rasmussen, C. E., Gaussian Processes for Machine Learning (2006), MIT Press Cambridge: MIT Press Cambridge MA · Zbl 1177.68165
[29] Jones, D. R., A taxonomy of global optimization methods based on response surfaces, J. Global Optim., 21, 345-383 (2001) · Zbl 1172.90492
[30] Brochu, E.; Cora, V. M.; De Freitas, N., A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning (2010)
[31] Shahriari, B.; Swersky, K.; Wang, Z.; Adams, R. P.; Freitas, N.d., Taking the human out of the loop: A review of Bayesian optimization, Proc. IEEE, 104, 148-175 (2016)
[32] Mockus, J.; Tiesis, V.; Zilinskas, A., The application of Bayesian methods for seeking the extremum, (Towards Global Optimization, Vol. 2 (1978)), 2 · Zbl 0394.90090
[33] Schonlau, M., Computer Experiments and Global Optimization (1997), University of Waterloo, (Ph.D. thesis)
[34] Jones, D. R.; Schonlau, M.; Welch, W. J., Efficient global optimization of expensive black-box functions, J. Global Optim., 13, 455-492 (1998) · Zbl 0917.90270
[35] Vazquez, E.; Bect, J., Convergence properties of the expected improvement algorithm with fixed mean and covariance functions, J. Statist. Plann. Inference, 140, 3088-3095 (2010) · Zbl 1419.62200
[36] Bull, A. D., Convergence rates of efficient global optimization algorithms, J. Mach. Learn. Res., 12, 2879-2904 (2011) · Zbl 1280.90094
[37] Koehler, J. R.; Owen, A. B., 9. Computer experiments, (Handbook of Statistics (1996), Elsevier), 261-308 · Zbl 0919.62089
[38] Forrester, A.; Sobester, A.; Keane, A., Engineering Design via Surrogate Modelling: A Practical Guide (2008), John Wiley & Sons
[39] Zimmermann, R., On the maximum likelihood training of gradient-enhanced spatial Gaussian processes, SIAM J. Sci. Comput., 35, A2554-A2574 (2013) · Zbl 1283.62201
[40] Ulaganathan, S.; Couckuyt, I.; Dhaene, T.; Degroote, J.; Laermans, E., Performance study of gradient-enhanced Kriging, Eng. Comput., 32, 15-34 (2016) · Zbl 07160033
[41] Zhang, J.; Taflanidis, A. A.; Medina, J. C., Sequential approximate optimization for design under uncertainty problems utilizing Kriging metamodeling in augmented input space, Comput. Methods Appl. Mech. Engrg., 315, 369-395 (2017)
[42] Han, Z.-H.; Görtz, S.; Zimmermann, R., Improving variable-fidelity surrogate modeling via gradient-enhanced kriging and a generalized hybrid bridge function, Aerosp. Sci. Technol., 25, 177-189 (2013)
[43] Lophaven, S. N.; Nielsen, H. B.; Søndergaard, J., DACE - a Matlab Kriging Toolbox, Version 2.0Technical Report IMM-TR-2002-12 (2002), Technical University of Denmark
[44] Wu, A.; Aoi, M. C.; Pillow, J. W., Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature (2018)
[45] Keane, A. J.; Nair, P. B., Computational Approaches for Aerospace Design: The Pursuit of Excellence (2005), John Wiley & Sons
[46] Dalbey, K. R., Efficient and Robust Gradient Enhanced Kriging EmulatorsTechnical Report SAND2013-7022 (2013), Sandia National Laboratories
[47] Wu, J.; Poloczek, M.; Wilson, A. G.; Frazier, P., Bayesian optimization with gradients, Adv. Neural Inf. Process. Syst., 5267-5278 (2017)
[48] Zhang, Y.; Kristensen, J.; Ghosh, S.; Vandeputte, T.; Tallman, J.; Wang, L., Finding maximum expected improvement for high-dimensional design optimization, (AIAA Aviation 2019 Forum (2019), American Institute of Aeronautics and Astronautics)
[49] McKay, M. D.; Beckman, R. J.; Conover, W. J., A comparison of three methods for selecting values of input variables in the analysis of output from a computer code, Technometrics, 21, 239-245 (1979) · Zbl 0415.62011
[50] Lizotte, D. J.; Greiner, R.; Schuurmans, D., An experimental methodology for response surface optimization methods, J. Global Optim., 53, 699-736 (2012) · Zbl 1259.90102
[51] Santner, T. J.; Williams, B. J.; Notz, W., The Design and Analysis of Computer Experiments (2018), Springer · Zbl 1405.62110
[52] Sóbester, A.; Leary, S. J.; Keane, A. J., On the design of optimization strategies based on global response surface approximation models, J. Global Optim., 33, 31-59 (2005) · Zbl 1137.90743
[53] Zhang, J.; Sanderson, A. C., JADE: Adaptive differential evolution with optional external archive, IEEE Trans. Evol. Comput., 13, 945-958 (2009)
[54] Huang, D.; Allen, T. T.; Notz, W. I.; Zeng, N., Global optimization of stochastic black-box systems via sequential kriging meta-models, J. Global Optim., 34, 441-466 (2006) · Zbl 1098.90097
[55] Hicks, R. M.; Henne, P. A., Wing design by numerical optimization, J. Aircr., 15, 407-412 (1978)
[56] Economon, T. D.; Palacios, F.; Copeland, S. R.; Lukaczyk, T. W.; Alonso, J. J., SU2: An open-source suite for multiphysics simulation and design, AIAA J., 54, 828-846 (2016)
[57] Forrester, A. I.J.; Keane, A. J.; Bressloff, N. W., Design and analysis of noisy computer experiments, AIAA J., 44, 2331-2339 (2006)
[58] de Baar, J. H.S.; Scholcz, T. P.; Dwight, R. P., Exploiting adjoint derivatives in high-dimensional metamodels, AIAA J., 53, 1391-1395 (2015)
[59] Lam, R.; Poloczek, M.; Frazier, P.; Willcox, K. E., Advances in Bayesian optimization with applications in aerospace engineering, (2018 AIAA Non-Deterministic Approaches Conference (2018), American Institute of Aeronautics and Astronautics)
[60] Dwight, R.; Han, Z.-H., Efficient uncertainty quantification using gradient-enhanced kriging, (50th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (2009), American Institute of Aeronautics and Astronautics)
[61] Bhattrai, S.; de Baar, J. H.S.; Neely, A. J., Efficient uncertainty quantification for a hypersonic trailing-edge flap, using gradient-enhanced kriging, Aerosp. Sci. Technol., 80, 261-268 (2018)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.