##
**Calculus of variations and optimal control theory. A concise introduction.**
*(English)*
Zbl 1239.49001

Princeton, NJ: Princeton University Press (ISBN 978-0-691-15187-8/hbk; 978-1-400-84264-3/ebook). xv, 235 p. (2012).

This nicely and carefully written textbook collects lecture notes for a graduate course on optimal control given by the author at the University of Illinois. As convincingly explained by the author himself in the preface, the book fulfils a need for an elementary introduction to the mathematical theory of optimal control for graduate students and engineers. Indeed, the book does not assume an extensive technical background in functional analysis or optimization, but mathematical rigour and correctness are never sacrificed, and the required notions are gradually introduced and motivated throughout the text. Another key contribution of the book is its historical perspective. Technical results are systematically presented by following the historical developments of the field, emphasizing connections with calculus of variations and functional analysis.

Chapters 1 to 3 collect preparatory material on differential calculus, finite and infinite dimensional optimization, first and second order conditions for optimality, and the introduction to optimal control theory as an outgrowth of calculus of variations. The objective of these chapters is to set the ground for the 50-page long chapter 4, which contains a detailed proof of the Pontryagin Maximum Principle (PMP), in the case of control constraints and variable end point, i.e., the state at final time is constrained to a given manifold. The PMP provides necessary conditions of optimality of a control law (allowing to distinguish candidates for optimality), and chapter 5 describes sufficient conditions via dynamic programming and the Hamilton-Jacobi-Bellman (HJB) Partial Differential Equation (PDE) satisfied by the value function (allowing to decide whether a candidate satisfying the PMP is actually optimal). Finally, chapter 6 treats more deeply and systematically specific cases of the finite and infinite horizon linear quadratic regulator problem, a convex optimal control problem where the objective function is quadratic and dynamics are linear. The differential and algebraic Riccati equations are then carefully derived from the PMP. Readers already familiar with linear systems theory will find these connections useful. More advanced material, not required for an introductory course on optimal control, can be found in sections 5.3 (viscosity solution of the HJB PDE), 7.1 (PMP on manifolds), 7.2 (PMP seen as characteristic equations for the HJB PDE), 7.3 (Riccati equations for robust and H-infinity control), 7.4 (PMP for hybrid control). Generally speaking, this reviewer believes that it is ambitious to prove rigorously the PMP in a graduate course on optimal control. As mentioned by the author himself on page 171, there is certainly no easy proof of the PMP with control and variable end point constraints, and the relatively lengthy proof described in the book is a comprehensive update of the proofs found in the historical references [L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze and E. F. Mishchenko, The mathematical theory of optimal processes. New York and London: Interscience Publishers, a division of John Wiley & Sons, Inc. (1962; Zbl 0102.32001)] and [M. Athans and P. L. Falb, Optimal control. Maidenhead, Berksh.: McGraw-Hill Publishing Company (1966; Zbl 0196.46303)].

The author is well known in the systems and control community for the clarity of its expositions, both oral and written. Consistently, this textbook is a very readable, pedagogical and lucid treatment of a potentially difficult and technical topic. It will undoubtedly fill a gap between original references on optimal control from the 1960s and more recent but also more advanced mathematically oriented books (typically focusing more on differential geometry or functional analysis than on engineering related material). The author’s choice is to closely follow historical developments, starting with the study of variational problems of the 17th century, and culminating with the theory of viscosity solutions of nonlinear PDEs in the 1980s. This perspective is fruitful, as this consistently motivates and explains the introduction of new mathematical concepts.

The targeted readership are graduate students in engineering, but scholars not familiar with optimal control theory and willing to learn more without investing too much in mathematical details, will also benefit from reading the book. This material can also be recommended as a preparatory reading for more advanced or more mathematically oriented books on optimal control, this reviewer’s favourites including [A. Bressan and B. Piccoli, Introduction to the mathematical theory of control. AIMS Series on Applied Mathematics 2. Springfield, MO: American Institute of Mathematical Sciences (AIMS) (2007; Zbl 1127.93002)], [E. Trélat, Contrôle optimal: théorie et applications. 2nd ed., Vuibert, Paris (2008)], [J.-B. Hiriart-Urruty, Les mathématiques du mieux faire, vol. 2: la commande optimale pour les débutants. Opuscules. Paris: Ellipses (2008; Zbl 1190.49001)] and [L. C. Evans, An introduction to mathematical optimal control theory. Lecture notes, Dept. Math., Univ. California at Berkeley, version 0.2 available online in May (2012)], see also Section 10.3 of [L. C. Evans, Partial differential equations. 2nd ed., Graduate Studies in Mathematics 19. Providence, RI: American Mathematical Society (2010; Zbl 1194.35001)]. Finally, it must be mentioned that a preliminary version of the book is freely available for download in electronic format from the author’s webpage, a welcome initiative that should help disseminate further this excellent material to a broad readership.

Chapters 1 to 3 collect preparatory material on differential calculus, finite and infinite dimensional optimization, first and second order conditions for optimality, and the introduction to optimal control theory as an outgrowth of calculus of variations. The objective of these chapters is to set the ground for the 50-page long chapter 4, which contains a detailed proof of the Pontryagin Maximum Principle (PMP), in the case of control constraints and variable end point, i.e., the state at final time is constrained to a given manifold. The PMP provides necessary conditions of optimality of a control law (allowing to distinguish candidates for optimality), and chapter 5 describes sufficient conditions via dynamic programming and the Hamilton-Jacobi-Bellman (HJB) Partial Differential Equation (PDE) satisfied by the value function (allowing to decide whether a candidate satisfying the PMP is actually optimal). Finally, chapter 6 treats more deeply and systematically specific cases of the finite and infinite horizon linear quadratic regulator problem, a convex optimal control problem where the objective function is quadratic and dynamics are linear. The differential and algebraic Riccati equations are then carefully derived from the PMP. Readers already familiar with linear systems theory will find these connections useful. More advanced material, not required for an introductory course on optimal control, can be found in sections 5.3 (viscosity solution of the HJB PDE), 7.1 (PMP on manifolds), 7.2 (PMP seen as characteristic equations for the HJB PDE), 7.3 (Riccati equations for robust and H-infinity control), 7.4 (PMP for hybrid control). Generally speaking, this reviewer believes that it is ambitious to prove rigorously the PMP in a graduate course on optimal control. As mentioned by the author himself on page 171, there is certainly no easy proof of the PMP with control and variable end point constraints, and the relatively lengthy proof described in the book is a comprehensive update of the proofs found in the historical references [L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze and E. F. Mishchenko, The mathematical theory of optimal processes. New York and London: Interscience Publishers, a division of John Wiley & Sons, Inc. (1962; Zbl 0102.32001)] and [M. Athans and P. L. Falb, Optimal control. Maidenhead, Berksh.: McGraw-Hill Publishing Company (1966; Zbl 0196.46303)].

The author is well known in the systems and control community for the clarity of its expositions, both oral and written. Consistently, this textbook is a very readable, pedagogical and lucid treatment of a potentially difficult and technical topic. It will undoubtedly fill a gap between original references on optimal control from the 1960s and more recent but also more advanced mathematically oriented books (typically focusing more on differential geometry or functional analysis than on engineering related material). The author’s choice is to closely follow historical developments, starting with the study of variational problems of the 17th century, and culminating with the theory of viscosity solutions of nonlinear PDEs in the 1980s. This perspective is fruitful, as this consistently motivates and explains the introduction of new mathematical concepts.

The targeted readership are graduate students in engineering, but scholars not familiar with optimal control theory and willing to learn more without investing too much in mathematical details, will also benefit from reading the book. This material can also be recommended as a preparatory reading for more advanced or more mathematically oriented books on optimal control, this reviewer’s favourites including [A. Bressan and B. Piccoli, Introduction to the mathematical theory of control. AIMS Series on Applied Mathematics 2. Springfield, MO: American Institute of Mathematical Sciences (AIMS) (2007; Zbl 1127.93002)], [E. Trélat, Contrôle optimal: théorie et applications. 2nd ed., Vuibert, Paris (2008)], [J.-B. Hiriart-Urruty, Les mathématiques du mieux faire, vol. 2: la commande optimale pour les débutants. Opuscules. Paris: Ellipses (2008; Zbl 1190.49001)] and [L. C. Evans, An introduction to mathematical optimal control theory. Lecture notes, Dept. Math., Univ. California at Berkeley, version 0.2 available online in May (2012)], see also Section 10.3 of [L. C. Evans, Partial differential equations. 2nd ed., Graduate Studies in Mathematics 19. Providence, RI: American Mathematical Society (2010; Zbl 1194.35001)]. Finally, it must be mentioned that a preliminary version of the book is freely available for download in electronic format from the author’s webpage, a welcome initiative that should help disseminate further this excellent material to a broad readership.

Reviewer: Didier Henrion (Toulouse)