A new two-step gradient-type method for large-scale unconstrained optimization. (English) Zbl 1198.90395

Farid, Mahboubeh; Leong, Wah June; Hassan, Malik Abu

Comput. Math. Appl. 59, No. 10, 3301-3307 (2010).

Summary: We propose some improvements on a new gradient-type method for solving large-scale unconstrained optimization problems, in which we use data from two previous steps to revise the current approximate Hessian. The new method which we considered, resembles to that of Barzilai and Borwein (BB) method. The innovation features of this approach consist in using approximation of the Hessian in diagonal matrix form based on the modified weak secant equation rather than the multiple of the identity matrix in the BB method. Using this approach, we can obtain a higher order accuracy of Hessian approximation when compares to other existing BB-type method. By incorporating a simple monotone strategy, the global convergence of the new method is achieved. Practical insights into the effectiveness of the proposed method are given by numerical comparison with the BB method and its variant.

MSC:

90C52	Methods of reduced gradient type
65K10	Numerical optimization and variational techniques

Keywords:

diagonal updating; weak secant equation; two-step gradient method; Barzilai and Borwein method

Software:

minpack

References:

[1]	Barzilai, J.; Borwein, J.M., Two point step size gradient methods, IMA J. numer. anal., 8, 141-148, (1988) · Zbl 0638.65055
[2]	Akaike, H., On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method, Ann. inst. statist. math., 11, 1-17, (1959) · Zbl 0100.14002
[3]	Dai, Y.H.; Liao, L.Z., \(R\)-linear convergence of the Barzilai and Borwein gradient method, IMA J. numer. anal., 22, 1-10, (2002) · Zbl 1002.65069
[4]	Bin, Z.; Gao, L.; Dai, Y.H., Monotone projected gradient methods for large-scale box-constrained quadratic programming, Sci. China ser. A, 49, 688-702, (2006) · Zbl 1112.90056
[5]	Dai, Y.H.; Fletcher, R., Projected barzilai – borwein methods for large-scale box-constrained quadratic programming, Numer. math., 100, 21-47, (2005) · Zbl 1068.65073
[6]	Raydan, M., On the Barzilai and Borwein choice of steplength for the gradient method, IMA J. numer. anal., 13, 618-622, (1993)
[7]	Hassan, M.A.; Leong, W.J.; Farid, M., A new gradient method via quasi-Cauchy relation which guarantees descent, J. comput. appl. math., 230, 300-305, (2009) · Zbl 1179.65067
[8]	Leong, W.J.; Hassan, M.A.; Farid, M., A monotone gradient method via weak secant equation for unconstrained optimization, Taiwanese J. math., 14, 2, 413-423, (2010) · Zbl 1203.90148
[9]	Ford, J.A., Implicit updates in multi-step quasi-Newton methods, Comput. math. appl., 42, 1083-1091, (2001) · Zbl 0989.65063
[10]	Ford, J.A.; Moghrabi, L.A., Multi-step quasi-Newton methods for optimization, J. comput. appl. math., 50, 305-323, (1994) · Zbl 0807.65062
[11]	Ford, J.A.; Moghrabi, L.A., Alternating multi-step quasi-Newton methods for unconstrained optimization, J. comput. appl. math., 82, 105-116, (1997) · Zbl 0886.65064
[12]	Ford, J.A.; Thrmlikit, S., New implicite updates in multi-step quasi-Newton methods for unconstrained optimization, J. comput. appl. math., 152, 133-146, (2003) · Zbl 1025.65035
[13]	Andrei, N., An unconstrained optimization test functions collection, Adv. model. optim., 10, 147-161, (2008) · Zbl 1161.90486
[14]	Moré, J.J.; Garbow, B.S.; Hillstorm, K.E., Testing unconstrained optimization software, ACM trans. math. softw., 7, 17-41, (1981) · Zbl 0454.65049
[15]	Dolan, E.D.; Moré, J.J., Benchmarking optimization software with perpormance profiles, Math. program., 91, 201-213, (2002) · Zbl 1049.90004
[16]	Dai, Y.H.; Yuan, J.Y.; Yuan, Y., Modified two-point stepsize gradient methods for unconstrained optimization, Comput. optim. appl., 22, 103-109, (2002) · Zbl 1008.90056
[17]	Dai, Y.H.; Yuan, Y., Alternative minimization gradient method, IMA J. numer. anal., 23, 373-393, (2003)