A hybrid simplex search and particle swarm optimization for unconstrained optimization. (English) Zbl 1121.90116

Summary: This paper proposes the hybrid NM-PSO algorithm based on the Nelder-Mead (NM) simplex search method and particle swarm optimization (PSO) for unconstrained optimization. NM-PSO is very easy to implement in practice since it does not require gradient computation. The modification of both the Nelder–Mead simplex search method and particle swarm optimization intends to produce faster and more accurate convergence. The main purpose of the paper is to demonstrate how the standard particle swarm optimizers can be improved by incorporating a hybridization strategy. In a suite of 20 test function problems taken from the literature, computational results via a comprehensive experimental study, preceded by the investigation of parameter selection, show that the hybrid NM-PSO approach outperforms the other three relevant search techniques (i.e., the original NM simplex search method, the original PSO and the guaranteed convergence particle swarm optimization (GCPSO)) in terms of solution quality and convergence rate. In a later part of the comparative experiment, the NM-PSO algorithm is compared to various most up-to-date cooperative PSO (CPSO) procedures appearing in the literature. The comparison report still largely favors the NM-PSO algorithm in the performance of accuracy, robustness and function evaluation. As evidenced by the overall assessment based on two kinds of computational experience, the new algorithm has demonstrated to be extremely effective and efficient at locating best-practice optimal solutions for unconstrained optimization.


90C30 Nonlinear programming
90C59 Approximation methods and heuristics in mathematical programming


Full Text: DOI


[1] Nash, S.G.; Sofer, A., Linear and nonlinear programming, (1996), McGraw-Hill New York
[2] Nelder, J.A.; Mead, R., A simplex method for function minimization, Computer journal, 7, 308-313, (1965) · Zbl 0229.65053
[3] Olsson, D.M.; Nelson, L.S., The nelder – mead simplex procedure for function minimization, Technometrics, 17, 45-51, (1975) · Zbl 0295.65043
[4] Chen, D.H.; Saleem, Z.; Grace, D.W., A new simplex procedure for function minimization, International journal of modeling and simulation, 6, 81-85, (1986)
[5] R.C. Eberhart, J. Kennedy, A new optimizer using particle swarm theory, in: Proceedings of the Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, 1995, pp. 39-43.
[6] J. Kennedy, R.C. Eberhart, Particle swarm optimization, in: Proceedings of the IEEE International Conference on Neural Networks, Piscataway, NJ, USA, 1995, pp. 1942-1948.
[7] S. Smith, The simplex method and evolutionary algorithms, in: Proceedings of the IEEE International Conference on Evolutionary Computation 1998, pp. 799-804.
[8] Hooke, R.; Jeeves, T.A., Direct search solution of numerical and statistical problems, Journal of association for computing machinery, 8, 212-221, (1961) · Zbl 0111.12501
[9] F. van den Bergh, An analysis of particle swarm optimizers, Ph.D. dissertation, University of Pretoria, Pretoria, 2001.
[10] Rosenbrock, H.H., An automatic method for finding the greatest or least value of a function, Computer journal, 3, 175-184, (1960)
[11] Spendley, W.; Hext, G.R.; Himsworth, F.R., Sequential application of simplex designs in optimization and evolutionary operation, Technometrics, 4, 441-461, (1962) · Zbl 0121.35603
[12] Balakrishnan, J.; Gunasekaran, M.K.; Gopal, E.S.R., Critical dielectric – constant measurements in the binary liquid system – methanol + normal heptane, International journal of PA physics, 22, 286-298, (1984)
[13] Busing, W.R.; Matsui, M., The application of external forces to computational model of crystals, Acta crystallographica section A, 40, 532-540, (1984)
[14] Schulze, W.; Rehder, U., Organization and morphogenesis of the human seminiferous epithelium, Cellular tissue review, 237, 395-417, (1984)
[15] Silver, G.L., Space modification: an alternative approach to chemistry problem involving geometry, Journal of computational chemistry, 2, 478-490, (1981)
[16] Sthapit, P.R.; Ottoway, J.M.; Fell, G.S., Determination of lead in matural and TAP waters by flame atomic-fluorescence spectrometry, Analyst, 109, 1061-1075, (1984)
[17] Fletcher, R., Practical methods of optimization, (1987), John Wiley & Sons Chichester · Zbl 0905.65002
[18] Barton, R.R.; Ivey, J.S., Nelder – mead simplex modifications for simulation optimization, Management science, 42, 954-973, (1996) · Zbl 0884.90118
[19] Nazareth, L.; Tzeng, P., Gilding the lily: A variant of the nelder – mead algorithm based on Golden-section search, Computational optimization and applications, 22, 133-134, (2002) · Zbl 1007.90062
[20] X. Hu, R.C. Eberhart, Adaptive particle swarm optimization: Detection and response to dynamic systems, in: Proceedings of the IEEE International Conference on Evolutionary Computation, Honolulu, Hawaii, USA, 2002, pp. 1666-1670.
[21] R.C. Eberhart, Y. Shi, Tracking and optimizing dynamic systems with particle swarms, in: Proceedings of Congress on Evolutionary Computation, Seoul, Korea, 2001, pp. 94-97.
[22] X. Hu, R.C. Eberhart, Tracking dynamic systems with PSO: Where’s the cheese? in: Proceedings of The Workshop on Particle Swarm Optimization, Indianapolis, IN, USA, 2001.
[23] V. Tandon, Closing the gap between CAD/CAM and optimized CNC and milling, Master’s thesis, Purdue School of Engineering and Technology, Indianapolis, IN, USA, 2000.
[24] H. Yoshida, K. Kawata, Y. Fukuyama, Y. Nakanishi, A particle swarm optimization for reactive power and voltage control considering voltage stability, in: Proceedings of the International Conference on Intelligent System Application to Power Systems, Rio de Janeiro, Brazil, 1999, pp. 117-121.
[25] Brandstatter, B.; Baumgartner, U., Particle swarm optimization—mass-spring system analogon, IEEE transactions on magnetics, 38, 997-1000, (2002)
[26] R.C. Eberhart, X. Hu, Human tremoe analysis using particle swarm optimization, in: Proceedings of the Congress on Evolutionary Computation, Washington DC, USA, 1999, pp. 1927-1930.
[27] van den Bergh, F.; Engelbrecht, A.P., Cooperative learning in neural networks using particle swarm optimizers, South african computer journal, 26, 84-90, (2000)
[28] Clerc, M.; Kennedy, J., The particle swarm-explosion, stability and convergence in a multidimensional complex space, IEEE transactions on evolutionary computation, 6, 58-73, (2002)
[29] F. van den Bergh, A.P. Engelbrecht, A new locally convergent particle swarm optimizer, in: IEEE International Conference on Systems, Man and Cybernetics, 2002, pp. 96-101.
[30] Renders, J.M.; Flasse, S.P., Hybrid methods using genetic algorithms for global optimization, IEEE transaction on systems, man, and cybernetics—part B: cybernetics, 26, 243-258, (1996)
[31] Yen, J.; Liao, J.C.; Lee, B.; Randolph, D., A hybrid approach to modeling metabolic systems using a genetic algorithm and simplex method, IEEE transaction on systems, man, and cybernetics—part B: cybernetics, 28, 173-191, (1998)
[32] Box, G.E.P., Evolutionary operation: A method for increasing industrial productivity, Applied statistics, 6, 81-101, (1957)
[33] N. Higashi, H. Iba, Particle swarm optimization with Gaussian mutation, in: IEEE Proceedings on Swarm Intelligence Symposium, Indianapolis, USA, 2003, pp. 72-79.
[34] Moré, J.J.; Garbow, B.S.; Hillstrom, K.E., Testing unconstrained optimization software, ACM transactions on mathematical software, 7, 17-41, (1981) · Zbl 0454.65049
[35] Dennis, J.E.; Woods, D.J., Optimization on microcomputers: the nelder – mead simplex algorithm, (), 116-122
[36] van den Bergh, F.; Engelbrecht, A.P., A cooperative approach to particle swarm optimization, IEEE transactions on evolutionary computation, 8, 225-239, (2004)
[37] Salomon, R., Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions, Biosystems, 39, 263-278, (1996)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.