×
Shi, Zhenjun

Convergence of line search methods for unconstrained optimization. (English) Zbl 1072.65087

Appl. Math. Comput. 157, No. 2, 393-405 (2004).
Author’s abstract: Line search methods are traditional and successful methods for solving unconstrained optimization problems. Its convergence has attracted more attention in recent years. In this paper we analyze the general results on convergence of line search methods with seven line search rules. It is clarified that the search direction plays a main role in these methods and that step-size guarantees the global convergence in some cases. It is also proved that many line search methods have same convergence property. These convergence results can enable us to design powerful, effective, and stable algorithms in practice. Finally, a class of special line search methods is investigated.
Reviewer: Berwin A. Turlach (Crawley)

Cited in 37 Documents

MSC:

65K05 Numerical mathematical programming methods
90C30 Nonlinear programming

Keywords:

unconstrained minimization; line search method; convergence; algorithms
PDF BibTeX XML Cite
\textit{Z. Shi}, Appl. Math. Comput. 157, No. 2, 393--405 (2004; Zbl 1072.65087)
Full Text: DOI

References:

[1] Armijo, L., Minimization of function having Lipschitz continuous first partial derivatives, Pac. J. Math, 16, 1-3 (1966) · Zbl 0202.46105
[2] Bertsekas, D. P.; Tsitsiklis, J. N., Gradient convergence in gradient methods with errors, SIAM J. Optim, 10, 3, 627-642 (2000) · Zbl 1049.90130
[3] Bertsekas, D. P., Constrained Optimization and Lagrange Multiplier Methods (1982), Academic Press · Zbl 0453.65045
[4] Cohen, A. I., Stepsize analysis for descent methods, J. Optim. Theory Appl, 33, 2, 187-205 (1981) · Zbl 0421.49030
[5] Dussault, J. P., Convergence of implementable descent algorithms for unconstrained optimization, J. Optim. Theory Appl, 104, 3, 749-750 (2000) · Zbl 0974.90018
[6] Dai, Y. H.; Han, J.; Liu, G.; Sun, D.; Yin, H.; Yuan, Y.-X., Convergence properties of nonlinear conjugate gradient methods, SIAM J. Optim, 10, 2, 345-358 (1999) · Zbl 0957.65062
[7] Evtushenko, Y. G., Numerical Optimization Techniques (1985), Optimization Software: Optimization Software New York
[8] Gaviano, M.; Lera, D., On linear convergence of gradient-type minimization algorithms, J. Optim. Theory Appl, 98, 2, 475-487 (1998) · Zbl 0908.90208
[9] Grapsa, T. N.; Vrahatis, M. N., A dimensional-reducing method for unconstrained optimization, J. Comput. Appl. Math, 66, 239-253 (1996) · Zbl 0856.65074
[10] Hestenes, M. R., Conjugate Direction Methods in Optimization (1980), Springer-Verlag: Springer-Verlag New York · Zbl 0421.49033
[11] Liu, G. H.; Jing, L. L.; Han, L. X.; Han, D., A class of nonmonotone conjuagte gradient methods for unconstrained optimization, J. Optim. Theory Appl, 101, 1, 127-140 (1999) · Zbl 0956.90047
[12] Lemarechal, C.; Mifflin, R., Global and superlinear convergence of an algorithm for one-dimensional minimization of convex functions, Math. Program, 24, 241-256 (1982) · Zbl 0505.90064
[13] Powell, M. J.D., Direct search algorithms for optimization calculations, Acta Numer, 7, 287-336 (1998) · Zbl 0911.65050
[14] Moguerza, J. M.; Prieto, F. J., Combining search directions using gradient flows, Math. Program, 96, 3, 529-559 (2003) · Zbl 1049.90126
[15] Mladineo, R. H., Convergence rate of a global optimization algorithm, Math. Program, 54, 223-232 (1992) · Zbl 0756.90082
[16] Nocedal, J.; Stephen, J. W., Numerical Optimization (1999), Springer-Verlag: Springer-Verlag New York · Zbl 0930.65067
[17] Nocedal, J., Theory of algorithms for unconstrained optimization, Acta Numer, 1, 199-242 (1992) · Zbl 0766.65051
[18] Ortega, J. M.; Rheinboldt, W. C., Iterative Solution Of Nonlinear Equation in Several Variables (1970), Academic Press · Zbl 0241.65046
[19] Potra, F. A.; Shi, Y., Efficient line search algorithm for unconstrained optimization, J. Optim. Theory Appl, 85, 3, 677-704 (1995) · Zbl 0831.90106
[20] Rockafellar, R. T., Convex Analysis (1970), Princeton University Press · Zbl 0229.90020
[21] Shi, Z. J., A new memory gradient method under exact line search, Asian Pac. J. Oper. Res, 20, 2 (2003) · Zbl 1165.90646
[22] Shi, Z. J., The supermemory gradient method with inexact line searches, J. Appl. Sci, 21, 3 (2003)
[23] Sanmatlas, S.; Vercher, E., A generalized conjugate gradient algorithm, J. Optim. Theory Appl, 98, 2, 489-502 (1998) · Zbl 0908.90242
[24] Todd, M. J., On convergence properties of algorithms for unconstrained minimization, IMA J. Numer. Anal, 9, 435-441 (1989) · Zbl 0686.65032
[25] Wei, Z.; Qi, L.; Jiang, H., Some convergence properties of descent methods, J. Optim. Theory Appl, 95, 1, 177-188 (1997) · Zbl 0890.90167
[26] Vrahatis, M. N.; Androulakis, G. S.; Lambrino, J. N.; Magoulas, G. D., A class of gradient unconstrained minimization algorithms with adaptive stepsize, J. Comput. Appl. Math, 114, 367-386 (2000) · Zbl 0958.65072
[27] Vrahatis, M. N.; Androulakis, G. S.; Manoussakis, G. E., A new unconstrained optimization method for imprecise function and gradient values, J. Math. Anal. Appl, 197, 586-607 (1996) · Zbl 0887.90166
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.