×

zbMATH — the first resource for mathematics

A diagonal quasi-Newton updating method for unconstrained optimization. (English) Zbl 1416.49025
Summary: A diagonal quasi-Newton updating algorithm is presented. The elements of the diagonal matrix approximating the Hessian are determined by minimizing both the size of the change from the previous estimate and the trace of the update, subject to the weak secant equation. Under mild classical assumptions, the convergence of the algorithm is proved to be linear. The diagonal quasi-Newton update satisfies the bounded deterioration property. Numerical experiments with 80 unconstrained optimization test problems of different structures and complexities prove that the suggested algorithm is more efficient and more robust than the steepest descent, Cauchy with Oren and Luenberger scaling algorithm in its complementary form and classical Broyden-Fletcher-Goldfarb-Shanno algorithm.

MSC:
49M15 Newton-type methods
65K05 Numerical mathematical programming methods
90C30 Nonlinear programming
Software:
OPTIMA; L-BFGS; tn
PDF BibTeX Cite
Full Text: DOI
References:
[1] Davidon, W.C.: Variable metric methods for minimization. Argonne national laboratory report, argonne il (1959)
[2] Wolfe, P., Convergence conditions for ascent methods, SIAM Rev., 11, 226-235, (1969) · Zbl 0177.20603
[3] Wolfe, P., Convergence conditions for ascent methods. II: Some corrections, SIAM Rev., 13, 185-188, (1971) · Zbl 0216.26901
[4] Dennis, JEJr; Moré, JJ, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comput., 28, 549-560, (1974) · Zbl 0282.65042
[5] Dennis, JEJr; Moré, JJ, Quasi-newton methods, motivation and theory, SIAM Rev., 19, 46-89, (1977) · Zbl 0356.65041
[6] Nocedal, J., Theory of algorithms for unconstrained optimization, Acta Numerica, 1, 1-37, (1992) · Zbl 0766.65051
[7] Dennis, J.E. Jr, Schnabel, R.B.: Numerical methods for unconstrained optimization and nonlinear equations. Prentice-hall, inc. Englewood cliffs NJ (1983)
[8] Fletcher, R.: Practical methods of optimization 1, Unconstrained optimization. Wiley, New York (1987) · Zbl 0905.65002
[9] Gill, P.E., Murray, W., Wright, M.H.: Practical optimization. Academic Press, London (1981) · Zbl 0503.90062
[10] Nocedal, J., Wright, S.J.: Numerical optimization (second edition). Springer Science + Business Media (2006) · Zbl 1104.65059
[11] Bartholomew-Biggs, M.: Nonlinear Optimization with Engineering Applications. Springer Optimization and its Applications, vol. 19. Springer Science Business Media (2008) · Zbl 1167.90001
[12] Andrei, N.: Continuous nonlinear optimization for engineering applications in GAMS technology. Springer Optimization and its Applications, vol. 121. Springer Science Business Media (2017) · Zbl 1395.90001
[13] Nocedal, J., Updating quasi-Newton matrices with limited storage, Math. Comput., 35, 773-782, (1980) · Zbl 0464.65037
[14] Nash, SG, Preconditioning of truncated-Newton methods, SIAM J. Sci. Stat. Comput., 6, 599-616, (1985) · Zbl 0592.65038
[15] Nazareth, JL, If quasi-Newton then why not quasi-Cauchy?, SIAG/Opt Views-and-news, 6, 11-14, (1995)
[16] Dennis, J.; Wolkowicz, H., Sizing and least-change secant methods. SIAM, J. Numer. Anal., 30, 1291-1314, (1993) · Zbl 0802.65081
[17] Zhu, M.; Nazareth, JL; Wolkowicz, H., The quasi-Cauchy relation and diagonal updating, SIAM J. Optim., 9, 1192-1204, (1999) · Zbl 1013.90137
[18] Leong, WJ; Farid, M.; Hassan, MA, Improved Hessian approximation with modified quasi-Cauchy relation for a gradient-type method, AMO-Adv Model. Optim., 12, 37-44, (2010) · Zbl 1332.90346
[19] Farid, M.; Leong, WJ; Zheng, L., A new diagonal gradient-type method for large scale unconstrained optimization, U.P.B. Sci. Bull. Series A, 75, 57-64, (2013) · Zbl 1299.65118
[20] Luenberger, D.G.: Linear and nonlinear programming, 2nd edn., Addison-Wesley, Reading, MA (1994) · Zbl 1134.90040
[21] Andrei, N., An unconstrained optimization test functions collection, Adv. Model. Optim.—An Electron. Int. J., 10, 147-161, (2008) · Zbl 1161.90486
[22] Broyden, CG; Dennis, JE; Moré, JJ, On the local and superlinear convergence of quasi-Newton methods, J. Inst. Math. Appl., 12, 223-246, (1973) · Zbl 0282.65041
[23] Dolan, ED; Moré, JJ, Benchmarking optimization software with performance profiles, Math. Program., 91, 201-213, (2002) · Zbl 1049.90004
[24] Amini, K.; Rizi, AG, A new structured quasi-Newton algorithm using partial information on Hessian, J. Comput. Appl. Math., 234, 805-811, (2010) · Zbl 1190.65094
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.