A new subspace minimization conjugate gradient method based on tensor model for unconstrained optimization. (English) Zbl 07474751

Summary: A new subspace minimization conjugate gradient method based on tensor model is proposed and analysed. If the objective function is close to a quadratic, we construct a quadratic approximation model in a two-dimensional subspace to generate the search direction; otherwise, we construct a tensor model. It is remarkable that the search direction satisfies the sufficient descent property. We prove the global convergence of the proposed method under mild assumptions. Numerical comparisons are given with well-known CGOPT and CG_DESCENT and show that the proposed algorithm is very promising.


90C30 Nonlinear programming
90C06 Large-scale problems in mathematical programming
65K05 Numerical mathematical programming methods
Full Text: DOI


[1] Andrei, N., An unconstrained optimization test functions collection, Adv. Model. Optim., 10, 147-161 (2008) · Zbl 1161.90486
[2] Andrei, N., An accelerated subspace minimization three-term conjugate gradient algorithm for unconstrained optimization, Numer. Algorithm., 65, 859-874 (2014) · Zbl 1301.65041
[3] Barzilai, J.; Borwein, J. M., Two-point step size gradient methods, IMA J. Numer. Anal., 8, 141-148 (1988) · Zbl 0638.65055
[4] Bouaricha, A., A software package for large, sparse unconstrained optimization using tensor methods, ACM T. Math. softw., 23, 81-90 (1997) · Zbl 0887.65066
[5] Bouaricha, A., Tensor methods for large sparse unconstrained optimization, SIAM J. Optim., 7, 732-756 (1997) · Zbl 0893.65042
[6] Bouaricha, A.; Schanabel, R. B., Tensor methods for large, sparse nonlinear least squares problems, SIAM J. Sci. Comput., 21, 1199-1221 (1999) · Zbl 0957.65065
[7] Chow, T. T.; Eskow, E.; Schnabel, R. B., Algorithm 738: A software package for unconstrained optimization using tensor methods, ACM T. Math. Softw., 20, 518-530 (1994) · Zbl 0888.65075
[8] Dai, Y. H.; Kou, C. X., A nonlinear conjugate gradient algorithm with an optimal property and an improved wolfe line search, SIAM J. Optim., 23, 296-320 (2013) · Zbl 1266.49065
[9] Dai, Y. H.; Kou, C. X., A Barzilai-Borwein conjugate gradient method, Sci. China Math., 59, 1511-1524 (2016) · Zbl 1352.49031
[10] Dai, Y. H.; Liao, L. Z., New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43, 87-101 (2001) · Zbl 0973.65050
[11] Dai, Y. H.; Yuan, Y., A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., 10, 177-182 (1999) · Zbl 0957.65061
[12] Dolan, E. D.; Mor, J. J., Benchmarking optimization software with performance profiles, Math. Programma., 91, 201-213 (2002) · Zbl 1049.90004
[13] Fatemi, M., A new efficient conjugate gradient method for unconstrained optimization, J. Comput. Appl. Math., 300, 207-216 (2016) · Zbl 1382.65170
[14] Flecher, R.; Reeves, C. M., Function minimization by conjugate gradient method, Comput. J., 7, 149-154 (1964) · Zbl 0132.11701
[15] Gould, N. I.M.; Orban, D.; Toint, Ph. L., CUTEst: A constrained and unconstrained testing environment with safe threads for mathematical optimization, Comput. Optim. Appl., 60, 545-557 (2015) · Zbl 1325.90004
[16] Hager, W. W.; Zhang, H., A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16, 170-192 (2005) · Zbl 1093.90085
[17] Hager, W. W.; Zhang, H., A survey of nonlinear conjugate gradient methods, Pac. J. Optim., 2, 35-58 (2006) · Zbl 1117.90048
[18] Hager, W. W.; Zhang, H., The limited memory conjugate gradient method, SIAM J. Optim., 23, 2150-2168 (2013) · Zbl 1298.90129
[19] Han, Q. M.; Sun, W. Y.; Han, J. Y., An adaptive conic trust-region method for unconstrained optimization, Optim. Methods. Softw., 20, 665-677 (2005) · Zbl 1127.90415
[20] Hestenes, M. R.; Stiefel, E., Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Stand., 49, 409-436 (1952) · Zbl 0048.09901
[21] Huang, D. Q.; Tang, Y. L.; Zhang, W. N., Distribution of roots of cubic equations, J. Korea Soc. Math. Educ. Ser. B Pure Appl. Math., 17, 185-188 (2010) · Zbl 1195.65065
[22] Li, L. B., A new algorithm for solving large scale trust region subproblem, Oper. Res. Manag. Sci., 16, 48-52 (2007)
[23] Li, M.; Liu, H. W.; Liu, Z. X., A new subspace minimization conjugate gradient method with nonmonotone line search for unconstrained optimization, Numer. Algorithm., 79, 195-219 (2018) · Zbl 1395.90225
[24] Liu, Z. X.; Liu, H. W., An efficient gradient method with approximate optimal stepsize for large scale unconstrained optimization, Numer. Algorithm., 78, 21-39 (2018) · Zbl 1397.90270
[25] Polak, E.; Ribire, G., Note sur la convergence de méthods de directions conjuguées, Rev. Franaise Informat. Rech. Opérationnelle., 3, 35-43 (1969) · Zbl 0174.48001
[26] Radosaw, P., Conjugate Gradient Algorithms in Nonconvex Optimization (2009), Springer-Verlag: Springer-Verlag, Berlin
[27] Rivaie, M.; Mamat, M.; Abashar, A., A new class of nonlinear conjugate gradient coefficients with exact and inexact line searches, Appl. Math. Comput., 268, 1152-1163 (2015) · Zbl 1410.90203
[28] Schnabel, R. B.; Chow, T. T., Tensor methods for unconstrained optimization using second derivatives, SIAM J. Optim., 1, 293-315 (1991) · Zbl 0758.65047
[29] Schnabel, R. B.; Frank, P. D., Tensor methods for nonlinear equations, SIAM J. Numer. Anal., 21, 815-843 (1984) · Zbl 0562.65029
[30] Shi, X.; Yang, L.; Zhang, Y., A non-monotone tensor method for unconstrained optimization problems, Wseas Trans. Math., 11, 1006-1017 (2012)
[31] Yang, Y. T.; Chen, Y. T.; Lu, Y. L., A subspace conjugate gradient algorithm for large-scale unconstrained optimization, Numer. Algorithm., 76, 813-828 (2017) · Zbl 1379.65038
[32] Yuan, Y., A modified BFGS algorithm for unconstrained optimization, IMA J. Numer. Anal., 11, 325-332 (1991) · Zbl 0733.65039
[33] Yuan, Y., Subspace methods for large scale nonlinear equations and nonlinear least squares, Optim. Eng., 10, 207-218 (2009) · Zbl 1171.65040
[34] Yuan, Y.; Stoer, J., A subspace study on conjugate gradient algorithms, Z. Angew. Math. Mech., 75, 69-77 (1995) · Zbl 0823.65061
[35] Zhang, H.; Hager, W. W., A nonmnotone line search technique and its application to unconstrained optimization, SIAM J. Optim., 14, 1043-1056 (2004) · Zbl 1073.90024
[36] Liu, H. W.; Liu, Z. X., An efficient Barzilai-Borwein conjugate gradient method for unconstrained optimization, J. Optim. Theory Appl. · Zbl 1409.49031
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.