Combined trust region methods for nonlinear least squares. (English) Zbl 0882.65053

Trust region realizations of the Gauss-Newton method are commonly used for obtaining solution of nonlinear least squares problems. The author proposes three efficient algorithms which improve standard trust region techniques: multiple dog-leg strategy for dense problems and two combined conjugate gradient Lanczos strategies for sparse problems. Efficiency of these methods is demonstrated by extensive numerical experiments.


65K05 Numerical mathematical programming methods
90C30 Nonlinear programming


minpack; GQTPAR
Full Text: EuDML Link


[1] M. Al-Baali R. Fletcher: Variational methods for nonlinear least squares. J. Optim. Theory Appl. 36 (1985), 405-421. · Zbl 0578.65064 · doi:10.2307/2582880
[2] R. H. Byrd R. B. Schnabel G. A. Shultz: Approximate solution of the trust region problem by minimization over two-dimensional subspaces. Math. Programming 40 (1988), 247-263. · Zbl 0652.90082 · doi:10.1007/BF01580735
[3] J. E. Dennis: Some computational techniques for the nonlinear least squares problem. Numerical solution of nonlinear algebraic equations (G. D. Byrne, C. A. Hall, Academic Press, London 1974.
[4] J. E. Dennis H. H. W. Mei: An Unconstrained Optimization Algorithm which Uses Function and Gradient Values. Research Report No. TR-75-246, Department of Computer Science, Cornell University 1975.
[5] J. E. Dennis D. M. Gay R. E. Welsch: An adaptive nonlinear least-squares algorithm. ACM Trans. Math. Software 7 (1981), 348-368. · Zbl 0464.65040 · doi:10.1145/355958.355965
[6] J. E. Dennis R. B. Schnabel: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs, New Jersey 1983. · Zbl 0579.65058
[7] R. Fletcher: A Modified Marquardt Subroutine for Nonlinear Least Squares. Research Report No.R-6799, Theoretical Physics Division, A.E.R.E. Harwell 1971.
[8] R. Fletcher: Practical Methods of Optimization. J. Wiley & Sons, Chichester 1987. · Zbl 0905.65002
[9] R. Fletcher C. Xu: Hybrid methods for nonlinear least squares. IMA J. Numer. Anal. 7 (1987), 371-389. · Zbl 0648.65051 · doi:10.1093/imanum/7.3.371
[10] P. E. Gill W. Murray: Newton type methods for unconstrained and linearly constrained optimization. Math. Programming 7 (1974), 311-350. · Zbl 0297.90082 · doi:10.1007/BF01585529
[11] G. H. Golub C. F. Van Loan: Matrix Computations. Johns Hopkins University Press, Baltimore 1989. · Zbl 0733.65016
[12] M. R. Hestenes: Conjugate Direction Methods in Optimization. Springer-Verlag, Berlin 1980. · Zbl 0439.49001
[13] K. Levenberg: A method for the solution of certain nonlinear problems in least squares. Quart. Appl. Math. 2 (1944), 164-168. · Zbl 0063.03501
[14] L. Lukšan: Inexact trust region method for large sparse nonlinear least squares. Kybernetika 29 (1993), 305-324. · Zbl 0806.65060
[15] L. Lukšan: Hybrid methods for large sparse nonlinear least squares. J. Optim. Theory Appl. 89 (1996), to appear. · Zbl 0851.90118
[16] D. W. Marquardt: An algorithm for least squares estimation of non-linear parameters. SIAM J. Appl. Math. 11 (1963), 431-441. · Zbl 0112.10505 · doi:10.1137/0111030
[17] J. J. Moré B. S. Garbow K. E. Hillström: Testing unconstrained optimization software. ACM Trans. Math. Software 7 (1981), 17-41. · Zbl 0454.65049
[18] J. J. Moré D. C. Sorensen: Computing a trust region step. SIAM J. Sci. Statist. Comput. 4 (1983), 553-572. · Zbl 0551.65042 · doi:10.1137/0904038
[19] M. J. D. Powell: A new algorithm for unconstrained optimization. Nonlinear Programming (J. B. Rosen, O. L. Mangasarian, K. Ritter, Academic Press, London 1970. · Zbl 0228.90043
[20] M. J. D. Powell: On the global convergence of trust region algorithms for unconstrained minimization. Math. Programming 29 (1984), 297-303. · Zbl 0569.90069 · doi:10.1007/BF02591998
[21] G. A. Shultz R. B. Schnabel R. H. Byrd: A family of trust-region-based algorithms for unconstrained minimization with strong global convergence properties. SIAM J. Numer. Anal. 22 (1985), 47-67. · Zbl 0574.65061 · doi:10.1137/0722003
[22] T. Steihaug: The conjugate gradient method and trust regions in large-scale optimization. SIAM J. Numer. Anal. 20 (1983), 626-637. · Zbl 0518.65042 · doi:10.1137/0720042
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.