×

zbMATH — the first resource for mathematics

Multivariate spectral gradient method for unconstrained optimization. (English) Zbl 1155.65046
The authors present the multivariate spectral gradient (MSG) method for solving unconstrained optimization problems. Combined with some quasi-Newton property the MSG method allows an individual adaptive stepsize along each coordinate direction, which guarantees that the method is finitely convergent for positive definite quadratics. Especially, it converges no more than two steps for positive definite quadratics with diagonal Hessian, and quadratically for objective functions with positive definite diagonal Hessian. Moreover, based on a nonmonotone line search, global convergence is established for the MSG algorithm.
Also a numerical study of the MSG algorithm compared with the global Barzilai-Borwein (GBB) algorithm is given. The search direction of the MSG method is close to that presented in the paper by M. N. Vrahatis, G. S. Androulakis, J. N. Lambrinos and G. D. Magoulas [J. Comput. App. Math. 114, 367–386 (2000; Zbl 0958.65072)], but the explanation for the steplength selection is different. The stepsize in this method is selected from the estimates of the eigenvalues of the Hessian but not a local estimation of the Lipschitz constant in the above mentioned paper. At last numerical results are reported, which show that this method is promising and deserves futher discussing.

MSC:
65K05 Numerical mathematical programming methods
90C30 Nonlinear programming
90C53 Methods of quasi-Newton type
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Barzilai, J.; Borwein, J.M., Two point step size gradient methods, IMA J. numer. anal., 8, 141-148, (1998) · Zbl 0638.65055
[2] Raydan, M., On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. numer. anal., 13, 321-326, (1993) · Zbl 0778.65045
[3] Raydan, M., The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM J. optim., 7, 26-33, (1997) · Zbl 0898.90119
[4] Grippo, L.; Lampariello, F.; Lucidi, S., A nonmonotone line search technique for newton’s method, SIAM J. numer. anal., 23, 707-716, (1986) · Zbl 0616.65067
[5] R. Fletcher, On the Barzilai-Borwein method, Numerical Analysis Report NA/207, October 2001. · Zbl 1118.90318
[6] Dai, Y.H.; Liao, L.Z., R-linear convergence of the Barzilai and Borwein gradient method, IMA J. numer. anal., 22, 1-10, (2002) · Zbl 1002.65069
[7] Molina, B.; Raydan, M., Preconditioned barzilai – borwein method for the numerical solution of partial differential equations, Numer. algor., 13, 45-60, (1996) · Zbl 0861.65025
[8] Dai, Y.H.; Yuan, J.Y.; Yuan, Y.X., Modified two-point stepsize gradient methods for unconstrained optimization, Comput. optim. appl., 22, 103-109, (2002) · Zbl 1008.90056
[9] Luengo, F.; Raydan, M.; Glunt, W.; Hayden, T.L., Preconditioned spectral gradient method, Numer. algor., 30, 241-258, (2002) · Zbl 1027.90110
[10] Dai, Y.H.; Zhang, H.C., Adaptive two-point stepsize gradient algorithm, Numer. algor., 27, 377-385, (2001) · Zbl 0992.65063
[11] Grippo, L.; Sciandrone, M., Nonmonotone globalization techniques for the barzilai – borwein gradient method, Comput. optim. appl., 23, 143-169, (2002) · Zbl 1028.90061
[12] Vrahatis, M.N.; Androulakis, G.S.; Lambrinos, J.N.; Magoulas, G.D., A class of gradient unconstrained minimization algorithms with adaptive stepsize, J. comput. appl. math., 114, 367-386, (2000) · Zbl 0958.65072
[13] Zhang, H.C.; Hager, W.W., A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. optim., 14, 1043-1056, (2004) · Zbl 1073.90024
[14] Dai, Y.H., On the nonmonotone line search, J. optim. theory appl., 112, 315-330, (2002) · Zbl 1049.90087
[15] Shi, Z.J.; Shen, J., Convergence of nonmonotone line search method, J. comput. appl. math., 193, 397-412, (2006) · Zbl 1136.90477
[16] Shi, Z.J., Convergence of line search methods for unconstrained optimization, Appl. math. comput., 157, 393-405, (2004) · Zbl 1072.65087
[17] Sun, W.; Han, J.; Sun, J., Global convergence of nonmonotone descent methods for unconstrained optimization problems, J. comput. appl. math., 146, 89-98, (2002) · Zbl 1007.65044
[18] Sun, W.Y., Nonmonotone trust region method for solving optimization problems, Appl. math. comput., 156, 159-174, (2004) · Zbl 1059.65055
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.