The cyclic Barzilai-Borwein method for unconstrained optimization. (English) Zbl 1147.65315

Dai, Yuhong; Hager, William W.; Schittkowski, Klaus; Zhang, Hongchao

IMA J. Numer. Anal. 26, No. 3, 604-627 (2006).

Summary: In the cyclic Barzilai-Borwein (CBB) method, the same Barzilai-Borwein (BB) stepsize is reused for \(m\) consecutive iterations. It is proved that CBB is locally linearly convergent at a local minimizer with positive definite Hessian. Numerical evidence indicates that when \(m>n/2\geq 3\), where \(n\) is the problem dimension, CBB is locally superlinearly convergent. In the special case \(m=3\) and \(n=2\), it is proved that the convergence rate is no better than linear, in general. An implementation of the CBB method, called adaptive cyclic Barzilai-Borwein (ACBB), combines a non-monotone line search and an adaptive choice for the cycle length \(m\). In numerical experiments using the CUTEr test problem library, ACBB performs better than the existing BB gradient algorithm, while it is competitive with the well-known PRP+ conjugate gradient algorithm.

MSC:

65K05	Numerical mathematical programming methods
90C20	Quadratic programming
90C25	Convex programming
90C30	Nonlinear programming

Keywords:

unconstrained optimization; gradient method; convex quadratic programming; non-monotone line search; cyclic Barzilai-Borwein method; convergence; numerical experiments; algorithm; conjugate gradient algorithm

Software:

CUTEr; CG_DESCENT