##
**Dynamic programming and Markov processes.**
*(English)*
Zbl 0091.16001

New York-London: John Wiley & Sons, Inc. and the Technology Press of the Massachusetts Institute of Technology. viii, 136 p. (1960).

This monograph consists of eight chapters. After giving an introduction to Markov process with particular reference to its \(z\)-transformation and to its rewards in the first two chapters, two approaches are given for solving sequential decision process associated with rewards. The first approach given in Chapter 3 is called “the solution by value iteration”, while the second one in Chapter 4 is “the policy-iteration method” consisting of the value-determination operation and the policy-improvement routine. Chapter 5 is devoted to illustrations of several examples. Chapter 6 explains the policy-iteration method for multiple chain process. Chapter 7 contains the sequential decision process with discounting. The last Chapter 8 is devoted to the continuous-time decision process.

In short the monograph presents us a new method for optimum decision in processes of long duration with clarity of presentation at the graduate level and at the same time with suggestion of further needs of researches in this important field of dynamic programming.

In short the monograph presents us a new method for optimum decision in processes of long duration with clarity of presentation at the graduate level and at the same time with suggestion of further needs of researches in this important field of dynamic programming.

Reviewer: T. Kitagawa

### MSC:

90C39 | Dynamic programming |

90C40 | Markov and semi-Markov decision processes |

90-02 | Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming |