*(English)*Zbl 1125.90056

This is the third edition of the text on dynamic programming and its applications. It introduces a large variety of ideas, facts, and algorithms for the general theory and applications. The latter include linear systems, inventory control, portfolio optimization, hypothesis testing, and many other applications. To avoid the use of measure theory and several mathematical complications, such as measurability and non-measurability of objective functions and the existence of measurable selectors, the ideas and methods are presented in a general form, but the presentation is rigorous mainly for the case of countable underlying probability spaces. The new edition contains a substantial amount of new material. In particular, new material on approximate dynamic programming was added and this topic became one of the principle focal points of the book. The major differences with the second edition of volume 1 include the following changes and additions:

(a) A new section “Dynamic Programming and Minimax Control was added to Chapter 1 “The Dynamic Programming Algorithm.” The material of this section is connected with new material in Chapter 6 ”Suboptimal Control.”

(b) The section on auction algorithms for shortest paths in Chapter 2 “Deterministic Systems and Shortest Path Problems” was eliminated.

(c) The section “Constrained and Multiobjective Problems” was added to Chapter 2.

(d) The material on sufficient statistics and partially observed Markov decision problems in Section 5.4 “Sufficient Statistics and Finite-State Markov Chains” was restructured and expanded.

(e) Significant new material on approximate methods was added to Section 6.

(f) New exercises were added.

##### MSC:

90C39 | Dynamic programming |

90-01 | Textbooks (optimization) |

49L20 | Dynamic programming method (infinite-dimensional problems) |

90C40 | Markov and semi-Markov decision processes |

93E20 | Optimal stochastic control (systems) |