×

On Hager and Zhang’s conjugate gradient method with guaranteed descent. (English) Zbl 1334.65111

Summary: We point out that Hager-Zhang family of conjugate gradient methods [W. W. Hager and H. Zhang, SIAM J. Optim. 16, No. 1, 170–192 (2005; Zbl 1093.90085)] are based on the spectral scaling secant equation, are really members of Dai-Liao type of gradient methods [Y. H. Dai and L. Z. Liao, Appl. Math. Optim. 43, No. 1, 87–101 (2001; Zbl 0973.65050)] with common constant parameters. We propose a possible efficient parameter for Hager-Zhang’s conjugate gradient method with guaranteed descent (CG_DESCENT) and present a conjugate gradient method CG_DESCENT1. The CG_DESCENT1 method preserves the same global convergence properties as CG_DESCENT method of Hager and Zhang, and is numerically tested compared with CG_DESCENT method and other conjugate gradient methods. Considering the difference of the search directions approximation to the search direction of the memoryless BFGS method, we give the reason why the proposed method is more efficient than CG_DESCENT method under some conditions, and further find an optimal conjugate gradient method and an approximation optimal conjugate method in Hager-Zhang family of conjugate gradient methods.

MSC:

65K10 Numerical optimization and variational techniques
90C52 Methods of reduced gradient type

Software:

CUTE; CG_DESCENT; CUTEr
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Barzilai, J.; Borwein, J. M., Two-point step size gradient method, IMA J. Numer. Anal., 8, 141-148 (1988) · Zbl 0638.65055
[2] Bongartz, I.; Conn, A. R.; Gould, N. I.M.; Toint, Ph. L., CUTE: constrained and unconstrained testing environment, ACM Trans. Math. Softw., 20, 123-160 (1995) · Zbl 0886.65058
[3] Cheng, W. Y.; Li, D. H., Spectral scaling BFGS method, J. Optim. Thoery Appl., 146, 305-319 (2010) · Zbl 1198.90355
[5] Dai, Y. H.; Liao, L. Z., New conjugacy conditions and related conjugate gradient methods, Appl. Math. Optim., 43, 87-101 (2001) · Zbl 0973.65050
[6] Dai, Y. H.; Ni, Q., Testing different conjugate gradient methods for large-scale unconstrained optimization, J. Comput. Math., 21, 311-320 (2003) · Zbl 1041.65048
[7] Dai, Y. H.; Yuan, Y. X., Nonlinear Conjugate Gradient Methods (2001), Shang Hai Science and Technolgy Press: Shang Hai Science and Technolgy Press Shanghai, (in Chinese) · Zbl 1030.90141
[8] Hager, W. W.; Zhang, H. C., A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16, 170-192 (2005) · Zbl 1093.90085
[9] Hager, W. W.; Zhang, H. C., Algorithm 851: CGDESCENT, a conjugate gradient method with guaranteed descent, ACM Trans. Math. Softw., 32, 113-137 (2006) · Zbl 1346.90816
[10] Hager, W. W.; Zhang, H. C., A survey of nonlinear gradient methods, Pacific J. Optim., 2, 35-58 (2006) · Zbl 1117.90048
[11] Kobayashi, M.; Narushima, Y.; Yabe, H., Nonlinear conjugate gradient methods with structured secant condition for nonlinear least squares problems, J. Comput. Appl. Math., 234, 375-397 (2010) · Zbl 1223.65043
[12] Li, D. H.; Fukushima, M., A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math., 129, 15-35 (2001) · Zbl 0984.65055
[13] Li, G. Y.; Tang, C. M.; Wei, Z. X., New conjugacy conditon and related new conjugate gradient methods for unconstrained optimization, J. Comput. Appl. Math., 202, 523-539 (2007) · Zbl 1116.65069
[14] Nocedal, J., Updating quasi-Newton matrixes with limited storage, Math. Comput., 35, 773-782 (1980) · Zbl 0464.65037
[15] Narushima, Y.; Yabe, H., Conjugate gradient methods on secant conditions that generate descent search directions for unconstrained optimization, J. Comput. Appl. Math., 236, 4303-4317 (2012) · Zbl 1258.65059
[16] Perry, J. M., A modified conjugate gradient algorithm, Oper. Res., 26, 1073-1078 (1978) · Zbl 0419.90074
[18] Shanno, D. F., On the convergence of a new conjugate gradient algorithm, SIAM J. Numer. Anal., 15, 1247-1257 (1978) · Zbl 0408.90071
[19] Wei, Z. X.; Li, G. Y.; Qi, L. Q., New quasi-Newton methods for unconstrained optimization problems, Appl. Math. Comput., 175, 1156-1188 (2006) · Zbl 1100.65054
[20] Yabe, H.; Takano, M., Globla convergence properties of nonlinear conjugate gradient methods with modified secant equation, Comput. Optim. Appl., 28, 203-225 (2004) · Zbl 1056.90130
[21] Zhang, J. Z.; Xu, C. X., Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations, J. Comput. Appl. Math., 137, 269-278 (2001) · Zbl 1001.65065
[22] Zhou, W. J.; Zhang, L., A nonlinear conjugate gradient method based on the MBFGS secant equation, Optim. Meth. Softw., 21, 707-714 (2006) · Zbl 1112.90096
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.