×

zbMATH — the first resource for mathematics

A new gradient method via quasi-Cauchy relation which guarantees descent. (English) Zbl 1179.65067
Authors’ abstract: We propose a new monotone algorithm for unconstrained optimization in the frame of J. Barzilai and J. Borwein (BB) method [IMA J. Numer. Anal. 8, 141–148 (1988; Zbl 0638.65055)] and analyze the convergence properties of this new descent method. Motivated by the fact that BB method does not guarantee descent in the objective function at each iteration, but performs better than the steepest descent method, we therefore attempt to find stepsize formula which enables us to approximate the Hessian based on the Quasi-Cauchy equation and possess monotone property in each iteration. Practical insights on the effectiveness of the proposed techniques are given by a numerical comparison with the BB method.

MSC:
65K05 Numerical mathematical programming methods
90C59 Approximation methods and heuristics in mathematical programming
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Akaike, H., On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method, Ann. inst. statist. math., Tokyo, 11, 1-17, (1959) · Zbl 0100.14002
[2] Barzilai, J.; Borwein, J.M., Two point step size gradient methods, IMA J. numer. anal., 8, 141-148, (1988) · Zbl 0638.65055
[3] Raydan, M., On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. numer. anal., 13, 618-622, (1993)
[4] Dai, Y.H.; Liao, L.Z., R-linear convergence of the Barzilai and Borwein gradient method, IMA J. numer. anal., 22, 1-10, (2002) · Zbl 1002.65069
[5] Dai, Y.H.; Yuan, J.Y.; Yuan, Y., Modified two-point stepsize gradient methods for unconstrained optimization, J. comput. optim. appl., 22, 103-109, (2002) · Zbl 1008.90056
[6] Dai, Y.H.; Yuan, Y., Alternative minimization gradient method, IMA J. numer. anal., 23, 373-393, (2003)
[7] R. Fletcher, On the Barzilai-Borwein method, Research Report NA/207, University of Dundee, UK, 2001 · Zbl 1118.90318
[8] Yuan, Y., A new stepsize for the steepest descent method, J. comput. math., 24, 149-156, (2006) · Zbl 1101.65067
[9] Dai, Y.H.; Fletcher, R., On the asymptotic behaviour of some new gradient methods, Math. program., 103, 541-559, (2005) · Zbl 1099.90038
[10] Han, L.; Yu, G.; Guan, L., Multivariate spectral gradient method for unconstrained optimization, Appl. math. comput., 201, 621-630, (2008) · Zbl 1155.65046
[11] Oren, S.; Spedicato, E., Optimal conditioning of self-scaling variable metric algorithms, Math. program., 10, 70-90, (1976) · Zbl 0342.90045
[12] Andrei, N., An unconstrained optimization test functions collection, J. amo, 10, 147-161, (2008) · Zbl 1161.90486
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.