A new gradient method via quasi-Cauchy relation which guarantees descent. (English) Zbl 1179.65067

Hassan, Malik Abu; Leong, Wah June; Farid, Mahboubeh

J. Comput. Appl. Math. 230, No. 1, 300-305 (2009).

Authors’ abstract: We propose a new monotone algorithm for unconstrained optimization in the frame of J. Barzilai and J. Borwein (BB) method [IMA J. Numer. Anal. 8, 141–148 (1988; Zbl 0638.65055)] and analyze the convergence properties of this new descent method. Motivated by the fact that BB method does not guarantee descent in the objective function at each iteration, but performs better than the steepest descent method, we therefore attempt to find stepsize formula which enables us to approximate the Hessian based on the Quasi-Cauchy equation and possess monotone property in each iteration. Practical insights on the effectiveness of the proposed techniques are given by a numerical comparison with the BB method.

Reviewer: Stefan Mititelu (Bucureşti)

MSC:

65K05	Numerical mathematical programming methods
90C59	Approximation methods and heuristics in mathematical programming

Keywords:

unconstrained optimization; monotone gradient methods; quasi-Cauchy relation; Barzilai and Borwein method

Citations:

Zbl 0638.65055

References:

[1]	Akaike, H., On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method, Ann. inst. statist. math., Tokyo, 11, 1-17, (1959) · Zbl 0100.14002
[2]	Barzilai, J.; Borwein, J.M., Two point step size gradient methods, IMA J. numer. anal., 8, 141-148, (1988) · Zbl 0638.65055
[3]	Raydan, M., On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. numer. anal., 13, 618-622, (1993)
[4]	Dai, Y.H.; Liao, L.Z., R-linear convergence of the Barzilai and Borwein gradient method, IMA J. numer. anal., 22, 1-10, (2002) · Zbl 1002.65069
[5]	Dai, Y.H.; Yuan, J.Y.; Yuan, Y., Modified two-point stepsize gradient methods for unconstrained optimization, J. comput. optim. appl., 22, 103-109, (2002) · Zbl 1008.90056
[6]	Dai, Y.H.; Yuan, Y., Alternative minimization gradient method, IMA J. numer. anal., 23, 373-393, (2003)
[7]	R. Fletcher, On the Barzilai-Borwein method, Research Report NA/207, University of Dundee, UK, 2001 · Zbl 1118.90318
[8]	Yuan, Y., A new stepsize for the steepest descent method, J. comput. math., 24, 149-156, (2006) · Zbl 1101.65067
[9]	Dai, Y.H.; Fletcher, R., On the asymptotic behaviour of some new gradient methods, Math. program., 103, 541-559, (2005) · Zbl 1099.90038
[10]	Han, L.; Yu, G.; Guan, L., Multivariate spectral gradient method for unconstrained optimization, Appl. math. comput., 201, 621-630, (2008) · Zbl 1155.65046
[11]	Oren, S.; Spedicato, E., Optimal conditioning of self-scaling variable metric algorithms, Math. program., 10, 70-90, (1976) · Zbl 0342.90045
[12]	Andrei, N., An unconstrained optimization test functions collection, J. amo, 10, 147-161, (2008) · Zbl 1161.90486