zbMATH — the first resource for mathematics

Markovian control problems. Functional equations and algorithms. Reprint. (English) Zbl 0541.90068
Mathematical Centre Tracts, 97. Amsterdam: Mathematisch Centrum. VII, 212 p. Dfl. 31.00 (1983).
This is a reprint of the author’s doctoral dissertation which was published in 1978. It consist of two parts: I. Markov decision theory and II. Stochastic games.
Part I is divided into five chapters: 1. Value iteration in finite Markov decision problems; 2. Contraction mappings underlying undiscounted Markov decision problems; 3. Nonstationary Markov decision problems with converging parameters; 4. Successive approximation methods for solving nested functional equations in Markov decision theory; 5. The optimality equation in average cost denumerable state semi-Markov decision problems, recurrence conditions and algorithms.
Part II is divided into three chapters: 6. On N-person stochastic games with denumerable state space; 7. On the functional equations in undiscounted and sensitive discounted stochastic games; 8. Successive approximation methods in two-person zero-sum stochastic games.
The list of references is up-to-dated apart from some papers (the most by the author or co-authored by himself). The treatment is self-contained and presents many important contributions due to the author. The book is very clearly written and makes genuine pleasant reading.
Reviewer: M.Iosifescu

90C40 Markov and semi-Markov decision processes
91A60 Probabilistic games; gambling
91A15 Stochastic games, stochastic differential games
91A05 2-person games
91A10 Noncooperative games
90-02 Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming