swMATH ID: 4829
Software Authors: Kelley, C.T.
Description: Iterative methods for optimization This book gives an introduction to optimization methods for unconstrained and bound constrained minimization problems. The style of the book is probably best described by the following quote from the book’s preface: ‘dots{} we treat a small number of methods in depth, giving less detailed description of only a few [dots{}]. We aim for clarity and brevity rather than complete generality and confine our scope to algorithms that are easy to implement (by the reader!) and understand.’ par This book is partitioned into two parts. The first part, occupying approximately 100 pages, is devoted to the optimization of smooth functions. The methods studied in this first part rely on the availability and accuracy of first order, and sometimes also second order derivatives of the objective function. The first part contains five chapters. The first chapter provides basic concepts. It also introduces a parameter identification problem and a discretized optimal control problem, both of which are used to demonstrate all methods discussed in the first part. Chapter 2 studies the local convergence of Newton’s method, inexact Newton methods, and the Gauss-Newton method for the solution of nonlinear least squares problems. Both, overdetermined and underdetermined nonlinear least squares problems are considered. Chapter 3 is devoted to line-search and trust-region methods, which are used to globalize convergence, i.e., remove the restriction that the starting point of the optimization iteration is sufficiently close to a solution. par The BFGS method is studied in chapter 4. A local convergence analysis is provided and implementation details are discussed. Other quasi-Newton methods are sketched. The last chapter of the first part, chapter 5, studies projection methods for the solution of bound constrained problems. All chapters conclude with a demonstration of the methods discussed in the respective chapter using the parameter identification problem and the discretized optimal control problem introduced in chapter 1, and with a set of exercises. par The second part of the book, which is approximately 50 pages long, deals with the optimization of noisy functions. Such optimization problems arise, e.g., when the evaluation of the objective function involves computer simulations. In such cases the noise often introduces artificial minimizers. Gradient information, even if available, cannot expected to be reliable. This second part contains three chapters. The first chapter provides a discussion of noisy functions, basics concepts, and three simple examples that are later used to demonstrate the behavior of optimization algorithms. Chapter 7 introduces implicit filtering, a technique due to the author and his group. Implicit filtering methods use finite difference approximations of the gradient, which are adjusted to the noise level in the function. Direct search algorithms, including the Nelder-Mead, multidirectional search, and the Hooke-Jeves algorithms are discussed in Chapter 8. Again, the latter two chapters conclude with a numerical demonstration of the methods discussed in the respective chapter, and with a set of exercises. par The treatment of both, optimization methods for smooth and for noisy functions is a unique feature of this book. Matlab implementations of all algorithms discussed in this book are easily accessible from the author’s or the publisher’s web-page.
Homepage: http://www4.ncsu.edu/~ctk/optimization_codes.html
Dependencies: Matlab
Keywords: unconstrained optimization; bound constraints; Newton’s method; Gauss-Newton method; quasi-Newton methods; line-search; trust-region; nonlinear least squares; projection methods; noisy functions; direct search methods; implicit filtering
Related Software: minpack; L-BFGS; Matlab; IMFIL; NITSOL; DFO; CUTEr; PETSc; NEWUOA; TRON; UOBYQA; mctoolbox; NewtonLib; SifDec; fminsearch; SDBOX; tn; LAPACK; L-BFGS-B; DFBOX_IMPR
Cited in: 664 Documents
all top 5

Cited by 1,170 Authors

24 Kelley, Carl T.
10 Bellavia, Stefania
10 Kevrekidis, Ioannis George
9 Banks, Harvey Thomas
9 Herty, Michael Matthias
8 Martínez, José Mario
8 Morini, Benedetta
8 Volkwein, Stefan
7 Antil, Harbir
7 Bardsley, Johnathan M.
7 Chacón, Luis
7 Fukushima, Masao
7 Haber, Eldad
7 Hedar, Abdel-Rahman
7 Olufsen, Mette Sofie
7 Siettos, Constantinos I.
6 Nagy, James Gerard
6 Ottesen, Johnny Tom
6 Quyen, Tran Nhan Tam
6 Vicente, Luis Nunes
5 Bai, Zhongzhi
5 Heinkenschloss, Matthias
5 Hinze, Michael
5 Kalita, Jiten C.
5 Klar, Axel
5 Pinnau, René
5 Seaïd, Mohammed
5 Yotov, Ivan
4 Bergamaschi, Luca
4 Blanco, Pablo Javier
4 Cătinaş, Emil
4 Chen, Ke
4 Coutinho, Alvaro L. G. A.
4 Galligani, Emanuele
4 Gear, Charles William
4 Hare, Warren L.
4 Jarry-Bolduc, Gabriel
4 Karátson, János
4 Kavanagh, Kathleen Rose
4 Lapenta, Giovanni
4 Macconi, Maria
4 Martínez, Ángeles
4 Martins, Marcos A. D.
4 Miller, Cass T.
4 Sabo, Kristian
4 Scitovski, Rudolf
4 Tran, Hien T.
4 Wheeler, Mary Fanett
3 Ascher, Uri M.
3 Banda, Mapundi Kondwani
3 Banerjee, Malay
3 Birgin, Ernesto G.
3 Bru, Rafael
3 Buscaglia, Gustavo Carlos
3 Diniz-Ehrhardt, Maria Aparecida
3 Duintjer Tebbens, Jurjen
3 Dussault, Jean-Pierre
3 Elman, Howard C.
3 Fan, Jinyan
3 Gablonsky, Joerg M.
3 Gomes-Ruggiero, Márcia Ap.
3 Gondzio, Jacek
3 Gugat, Martin
3 Hoppe, Ronald H. W.
3 Huang, Jingfang
3 Kees, Christopher E.
3 Klibanov, Michael V.
3 Krejić, Nataša
3 Lee, Taeyoung
3 Leiva, Jorge S.
3 Li, Zhaoyan
3 McClamroch, N. Harris
3 Mehlsen, Jesper
3 Planiden, Chayne
3 Reynolds, Daniel R.
3 Sachs, Ekkehard W.
3 Sciandrone, Marco
3 Tůma, Miroslav
3 Volpert, Vitaly A.
3 Xiao, Yunhai
3 Xin, Jianguo
3 Yuan, Ya-xiang
3 Zhang, Daoping
2 Ahookhosh, Masoud
2 Al-Baali, Mehiddin
2 Al-Jeiroudi, Ghussoun
2 Alves, José L. D.
2 Amini, Keyvan
2 An, Dong
2 Atkinson, Kendall Eugene
2 Bachmann, Freya
2 Baldick, Ross
2 Blauth, Sebastian
2 Brady, Renee
2 Caldwell, Timothy M.
2 Caliari, Marco
2 Calvetti, Daniela
2 Cardona, Alberto
2 Chan, Swee-Huat
2 Chan, Tony Fan-Cheong
...and 1,070 more Authors
all top 5

Cited in 212 Serials

28 Journal of Computational and Applied Mathematics
28 Computational Optimization and Applications
27 Journal of Computational Physics
22 Optimization Methods & Software
19 SIAM Journal on Scientific Computing
18 SIAM Journal on Optimization
17 Applied Mathematics and Computation
16 Computer Methods in Applied Mechanics and Engineering
16 Inverse Problems
13 Journal of Optimization Theory and Applications
12 Optimization and Engineering
10 International Journal for Numerical Methods in Engineering
10 Applied Numerical Mathematics
10 Numerical Algorithms
9 SIAM Journal on Numerical Analysis
9 Numerical Linear Algebra with Applications
8 Computer Physics Communications
8 Mathematics of Computation
7 Inverse Problems in Science and Engineering
6 Journal of Mathematical Biology
6 Automatica
6 Journal of Scientific Computing
5 Computers & Mathematics with Applications
5 Mathematics and Computers in Simulation
5 Numerical Functional Analysis and Optimization
5 Journal of Global Optimization
5 Computational Statistics and Data Analysis
5 SIAM Journal on Imaging Sciences
4 Computers and Fluids
4 International Journal for Numerical Methods in Fluids
4 Computing
4 Journal of Integral Equations and Applications
4 M\(^3\)AS. Mathematical Models & Methods in Applied Sciences
4 European Journal of Operational Research
4 SIAM Review
4 Mathematical Programming. Series A. Series B
4 Journal of Inverse and Ill-Posed Problems
4 SIAM Journal on Applied Dynamical Systems
4 Multiscale Modeling & Simulation
4 Acta Numerica
4 Inverse Problems and Imaging
3 IMA Journal of Numerical Analysis
3 Journal of Mathematical Analysis and Applications
3 Mathematical Biosciences
3 Bulletin of Mathematical Biology
3 Numerische Mathematik
3 SIAM Journal on Control and Optimization
3 Optimal Control Applications & Methods
3 Journal of Computational Mathematics
3 Numerical Methods for Partial Differential Equations
3 Journal of Economic Dynamics & Control
3 Applied Mathematics Letters
3 Mathematical and Computer Modelling
3 SIAM Journal on Matrix Analysis and Applications
3 International Journal of Computer Mathematics
3 Communications in Numerical Methods in Engineering
3 CEJOR. Central European Journal of Operations Research
3 Frontiers in Applied Mathematics
3 Optimization Letters
2 International Journal for Numerical and Analytical Methods in Geomechanics
2 ACM Transactions on Mathematical Software
2 Systems & Control Letters
2 Computational Mechanics
2 Journal of Parallel and Distributed Computing
2 IEEE Transactions on Circuits and Systems. I: Fundamental Theory and Applications
2 Journal of Non-Newtonian Fluid Mechanics
2 Journal of Mathematical Imaging and Vision
2 International Journal of Bifurcation and Chaos in Applied Sciences and Engineering
2 Electronic Journal of Differential Equations (EJDE)
2 ETNA. Electronic Transactions on Numerical Analysis
2 Advances in Computational Mathematics
2 Engineering Analysis with Boundary Elements
2 Mathematical Problems in Engineering
2 Computing and Visualization in Science
2 Multibody System Dynamics
2 Mathematical Methods of Operations Research
2 Communications in Nonlinear Science and Numerical Simulation
2 Computational Geosciences
2 Engineering Computations
2 International Journal of Computational Methods
2 Mathematical Biosciences and Engineering
2 European Series in Applied and Industrial Mathematics (ESAIM): Mathematical Modelling and Numerical Analysis
2 Pacific Journal of Optimization
2 Advances in Design and Control
2 Communications in Computational Physics
2 Mathematical Modelling of Natural Phenomena
2 Set-Valued and Variational Analysis
2 Journal of Theoretical Biology
2 Journal of Computational Dynamics
1 Applicable Analysis
1 Archive for Rational Mechanics and Analysis
1 Biological Cybernetics
1 International Journal of Control
1 International Journal of Engineering Science
1 International Journal of Plasticity
1 Indian Journal of Pure & Applied Mathematics
1 International Journal of Solids and Structures
1 Journal of Engineering Mathematics
1 Journal of Fluid Mechanics
...and 112 more Serials
all top 5

Cited in 45 Fields

384 Numerical analysis (65-XX)
195 Operations research, mathematical programming (90-XX)
118 Partial differential equations (35-XX)
91 Calculus of variations and optimal control; optimization (49-XX)
90 Fluid mechanics (76-XX)
62 Biology and other natural sciences (92-XX)
47 Computer science (68-XX)
43 Systems theory; control (93-XX)
37 Statistics (62-XX)
36 Mechanics of deformable solids (74-XX)
28 Information and communication theory, circuits (94-XX)
25 Optics, electromagnetic theory (78-XX)
24 Ordinary differential equations (34-XX)
19 Geophysics (86-XX)
16 Dynamical systems and ergodic theory (37-XX)
15 Game theory, economics, finance, and other social and behavioral sciences (91-XX)
13 Operator theory (47-XX)
12 Integral equations (45-XX)
12 Statistical mechanics, structure of matter (82-XX)
11 Classical thermodynamics, heat transfer (80-XX)
10 Probability theory and stochastic processes (60-XX)
9 Linear and multilinear algebra; matrix theory (15-XX)
8 Mechanics of particles and systems (70-XX)
7 Approximations and expansions (41-XX)
4 Combinatorics (05-XX)
4 Quantum theory (81-XX)
3 Astronomy and astrophysics (85-XX)
2 General and overarching topics; collections (00-XX)
2 Integral transforms, operational calculus (44-XX)
2 Functional analysis (46-XX)
2 Convex and discrete geometry (52-XX)
2 Differential geometry (53-XX)
2 Relativity and gravitational theory (83-XX)
1 History and biography (01-XX)
1 Field theory and polynomials (12-XX)
1 Commutative algebra (13-XX)
1 Group theory and generalizations (20-XX)
1 Real functions (26-XX)
1 Functions of a complex variable (30-XX)
1 Potential theory (31-XX)
1 Difference and functional equations (39-XX)
1 Sequences, series, summability (40-XX)
1 Harmonic analysis on Euclidean spaces (42-XX)
1 Geometry (51-XX)
1 Global analysis, analysis on manifolds (58-XX)

Citations by Year