zbMATH — the first resource for mathematics

Examples
Geometry Search for the term Geometry in any field. Queries are case-independent.
Funct* Wildcard queries are specified by * (e.g. functions, functorial, etc.). Otherwise the search is exact.
"Topological group" Phrases (multi-words) should be set in "straight quotation marks".
au: Bourbaki & ti: Algebra Search for author and title. The and-operator & is default and can be omitted.
Chebyshev | Tschebyscheff The or-operator | allows to search for Chebyshev or Tschebyscheff.
"Quasi* map*" py: 1989 The resulting documents have publication year 1989.
so: Eur* J* Mat* Soc* cc: 14 Search for publications in a particular source with a Mathematics Subject Classification code (cc) in 14.
"Partial diff* eq*" ! elliptic The not-operator ! eliminates all results containing the word elliptic.
dt: b & au: Hilbert The document type is set to books; alternatively: j for journal articles, a for book articles.
py: 2000-2015 cc: (94A | 11T) Number ranges are accepted. Terms can be grouped within (parentheses).
la: chinese Find documents in a given language. ISO 639-1 language codes can also be used.

Operators
a & b logic and
a | b logic or
!ab logic not
abc* right wildcard
"ab c" phrase
(ab c) parentheses
Fields
any anywhere an internal document identifier
au author, editor ai internal author identifier
ti title la language
so source ab review, abstract
py publication year rv reviewer
cc MSC code ut uncontrolled term
dt document type (j: journal article; b: book; a: book article)
Global convergence of a modified Fletcher-Reeves conjugate gradient method with Armijo-type line search. (English) Zbl 1103.65074
Summary: In this paper, we are concerned with the conjugate gradient methods for solving unconstrained optimization problems. It is well-known that the direction generated by a conjugate gradient method may not be a descent direction of the objective function. In this paper, we take a little modification to the Fletcher-Reeves (FR) method such that the direction generated by the modified method provides a descent direction for the objective function. This property depends neither on the line search used, nor on the convexity of the objective function. Moreover, the modified method reduces to the standard FR method if line search is exact. Under mild conditions, we prove that the modified method with Armijo-type line search is globally convergent even if the objective function is nonconvex. We also present some numerical results to show the efficiency of the proposed method.

MSC:
65K05Mathematical programming (numerical methods)
90C30Nonlinear programming
Software:
minpack; L-BFGS
References:
[1]Al-Baali M. (1985) Descent property and global convergence of the Fletcher–Reeves method with inexact line search. IMA J. Numer. Anal. 5, 121–124 · Zbl 0578.65063 · doi:10.1093/imanum/5.1.121
[2]Andrei, N.: Scaled conjugate gradient algorithms for unconstrained optimization. Comput. Optim. Appl. (to apper)
[3]Birgin E., Martínez J.M. (2001) A spectral conjugate gradient method for unconstrained optimization. Appl. Math. Optim. 43, 117–128 · Zbl 0990.90134 · doi:10.1007/s00245-001-0003-0
[4]Bongartz K.E., Conn A.R., Gould N.I.M., Toint P.L. (1995) CUTE: constrained and unconstrained testing environments. ACM Trans. Math. Softw. 21, 123–160 · Zbl 0886.65058 · doi:10.1145/200979.201043
[5]Dai Y.H., Yuan Y. (1996) Convergence properties of the conjugate descent method. Adv. Math. 25, 552–562
[6]Dai Y.H., Yuan Y. (1996) Convergence properties of the Fletcher–Reeves method. IMA J. Numer. Anal. 16, 155–164 · Zbl 0851.65049 · doi:10.1093/imanum/16.2.155
[7]Dai Y.H., Yuan Y. (2000) A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10, 177–182 · Zbl 0957.65061 · doi:10.1137/S1052623497318992
[8]Dai, Y.H.: Some new properties of a nonlinear conjugate gradient method. Research report ICM-98-010, Insititute of Computational Mathematics and Scientific/Engineering Computing. Chinese Academy of Sciences, Beijing (1998)
[9]Dixon L.C.W. (1970) Nonlinear optimization: a survey of the state of the art. In: Evans D.J. (eds) Software for Numerical Mathematics. Academic, New York, pp. 193–216
[10]Dolan E.D., Moré J.J. (2002) Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 · Zbl 1049.90004 · doi:10.1007/s101070100263
[11]Fletcher R. (1987) Practical Methods of Optimization, Unconstrained Optimization. vol. I. Wiley, New York
[12]Fletcher R., Reeves C. (1964) Function minimization by conjugate gradients. J. Comput. 7, 149–154 · Zbl 0132.11701 · doi:10.1093/comjnl/7.2.149
[13]Gilbert J.C., Nocedal J. (1992) Global convergence properties of conjugate gradient methods for optimization. SIAM. J. Optim. 2, 21–42 · Zbl 0767.90082 · doi:10.1137/0802003
[14]Grippo L., Lucidi S. (1997) A globally convergent version of the Polak–Ribiére gradient method. Math. Program. 78, 375–391
[15]Hager W.W., Zhang H. (2005) A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170–192 · Zbl 1093.90085 · doi:10.1137/030601880
[16]Hager W.W., Zhang H. (2006) A survey of nonlinear conjugate gradient methods. Pacific J. Optim. 2, 35–58
[17]Hestenes M.R., Stiefel E.L. (1952) Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. Sect. B. 49, 409–432
[18]Hu Y., Storey C. (1991) Global convergence results for conjugate gradient methods. J. Optim. Theory Appl. 71, 399–405 · Zbl 0794.90063 · doi:10.1007/BF00939927
[19]Liu D.C., Nocedal J. (1989) On the limited memory BFGS method for large scale optimization methods. Math. Program. 45, 503–528 · Zbl 0696.90048 · doi:10.1007/BF01589116
[20]Liu G., Han J., Yin H. (1995) Global convergence of the Fletcher–Reeves algorithm with inexact line search. Appl. Math. J. Chin. Univ. Ser. B. 10, 75–82 · Zbl 0834.90122 · doi:10.1007/BF02663897
[21]Moré J.J., Garbow B.S., Hillstrome K.E. (1981) Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 · Zbl 0454.65049 · doi:10.1145/355934.355936
[22]Moré J.J., Thuente D.J. (1994) Line search algorithms with guaranteed sufficient decrease. ACM Trans. Math. Softw. 20, 286–307 · Zbl 0888.65072 · doi:10.1145/192115.192132
[23]Nocedal J. (1996) Conjugate gradient methods and nonlinear optimization. In: Adams L., Nazareth J.L. (eds.) Linear and Nonlinear Conjugate Gradient-Related Methods. SIAM, Philadelphia, pp. 9–23
[24]Polak E. (1997) Optimization: Algorithms and Consistent Approximations. Springer, Berlin Heidelberg New York
[25]Polak B., Ribiere G. (1969) Note surla convergence des méthodes de directions conjuguées. Rev. Francaise Imformat Recherche Opertionelle 16, 35–43
[26]Polyak B.T. (1969) The conjugate gradient method in extreme problems. USSR Comp. Math. Math. Phys. 9, 94–112 · Zbl 0229.49023 · doi:10.1016/0041-5553(69)90035-4
[27]Powell M.J.D. (1984) Nonconvex minimization calculations and the conjugate gradient method. In: Lecture Notes in Mathematics 1066, 121–141
[28]Powell M.J.D. (1986) Convergence properties of algorithms for nonlinear optimization. SIAM Rev. 28, 487–500 · Zbl 0624.90091 · doi:10.1137/1028154
[29]Raydan M. (1997) The Barzilain and Borwein gradient method for the large unconstrained minimization problem. SIAM J. Optim. 7, 26–33 · Zbl 0898.90119 · doi:10.1137/S1052623494266365
[30]Sun J., Zhang J. (2001) Convergence of conjugate gradient methods without line search. Ann. Oper. Res. 103, 161–173 · Zbl 1014.90071 · doi:10.1023/A:1012903105391
[31]Touati-Ahmed D., Storey C. (1990) Efficient hybrid conjugate gradient techniques. J. Optim. Theory Appl. 64, 379–397 · Zbl 0687.90081 · doi:10.1007/BF00939455
[32]Zhang, L., Zhou, W.J., Li, D.H.: A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence. IMA J. Numer. Anal. (to appear)
[33]Zoutendijk G. (1970) Nonlinear programming, computational methods. In: Abadie J. (eds) Integer and Nonlinear Programming. North-Holland, Amsterdam, pp. 37–86