Hestenes, Magnus R. Multiplier and gradient methods. (English) Zbl 0174.20705 J. Optim. Theory Appl. 4, No. 5, 303-320 (1969). Summary: The main purpose of this paper is to suggest a method for finding the minimum of a function \(f(x)\) subject to the constraint \(g(x)=0\). The method consists of replacing \(f\) by \(F=f+\lambda g+ \tfrac12 cg^2\), where \(c\) is a suitably large constant, and computing the appropriate value of the Lagrange multiplier. Only the simplest algorithm is presented. The remaining part of the paper is devoted to a survey of known methods for finding unconstrained minima, with special emphasis on the various gradient techniques that are available. This includes Newton’s method and the method of conjugate gradients. Page: −5 −4 −3 −2 −1 ±0 +1 +2 +3 +4 +5 Show Scanned Page Cited in 518 Documents MSC: 65K10 Numerical optimization and variational techniques 49M15 Newton-type methods Keywords:minimum of a function subject to constraint; Lagrange multiplier; survey of known methods; finding unconstrained minima; gradient techniques; Newton’s method; method of conjugate gradients PDF BibTeX XML Cite \textit{M. R. Hestenes}, J. Optim. Theory Appl. 4, 303--320 (1969; Zbl 0174.20705) Full Text: DOI OpenURL References: [1] Hestenes, M. R.,A General Problem in the Calculus of Variations with Applications to Paths of Least Time, The RAND Corporation, Research Memorandum No. RM-100, 1950. [2] Mengel, A. S.,Optimum Trajectories, The RAND Corporation, Report No. P-199, 1951. [3] Hestenes, M. R.,Numerical Methods for Obtaining Solutions of Fixed-Endpoint Problems in the Calculus of Variations, The RAND Corporation, Research Memorandum No. RM-102, 1949. [4] Stein, M. L.,On Methods for Obtaining Solutions of Fixed-Endpoint Problems in the Calculus of Variations, Journal of Research of the National Bureau of Standards, Vol. 50, No. 5, 1953. · Zbl 0053.07503 [5] Hestenes, M. R.,Iterative Computational Methods, Communications on Pure and Applied Mathematics, Vol. 8, No. 1, 1955. · Zbl 0066.10201 [6] Balakrishnan, A. V.,On a New Computing Technique in Optimal Control and Its Application to Minimal-Time Flight Profile Optimization, Journal of Optimization Theory and Applications, Vol. 4, No. 1, 1969. · Zbl 0167.09101 [7] Hestenes, M. R.,An Indirect Sufficiency Proof for the Problem of Bolza in Nonparametric Form, Transactions of the American Mathematical Society, Vol. 62, No. 3, 1947. · Zbl 0032.02004 [8] Hestenes, M. R., andStiefel, E.,Methods of Conjugate Gradients for Solving Linear Systems, Journal of Research of the National Bureau of Standards, Vol. 49, No. 6, 1952. · Zbl 0048.09901 [9] Hestenes, M. R.,The Conjugate Gradient Method for Solving Linear Systems, Proceedings of the Sixth Symposium in Applied Mathematics, Edited by J. H. Curtiss, American Mathematical Society, Providence, Rhode Island, 1956. · Zbl 0072.14102 [10] Hayes, R. M.,Iterative Methods for Solving Linear Problems in Hilbert Space, Contributions to the Solutions of Systems of Linear Equations and the Determinations of Eigenvalues, Edited by O. Tausky, National Bureau of Standards, Applied Mathematics Series, US Government Printing Office, Washington, D.C., 1954. · Zbl 0058.10703 [11] Fletcher, R., andPowell, M. J. D.,A Rapidly Convergent Descent Method for Minimization, Computer Journal, Vol. 6, No. 2, 1964. · Zbl 0132.11603 [12] Myers, G. E.,Properties of the Conjugate-Gradient and Davidon Methods, Journal of Optimization Theory and Applications, Vol. 2, No. 4, 1968. · Zbl 0207.17302 [13] Horwitz, L. B., andSarachick, P. E.,Davidon’s Method in Hilbert Space, SIAM Journal on Applied Mathematics, Vol. 16, No. 4, 1968. · Zbl 0159.43803 This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.