zbMATH — the first resource for mathematics

Introduction to stochastic dynamic programming. (English) Zbl 0567.90065
Probability and Mathematical Statistics. New York - London etc.: Academic Press. A subsidiary of Harcourt Brace Jovanovich, Publishers. XI, 164 p. (1983).
This is an introduction into the basic models and techniques of stochastic dynamic programming (Markov decision processes) and its applications to operations research problems like sequential allocation, scheduling, and search. Only little knowledge in probability theory is needed. There are many exercises to each of the seven chapters (Finite- Stage Models, Discounted Dynamic Programming, Minimizing Costs - Negative Dynamic Programming, Maximizing Rewards - Positive Dynamic Programming, Average Reward Criterion, Stochastic Scheduling, Bandit Processes), and the Appendix (Stochastic Order Relations).
Reviewer: W.R.Heilmann

90C40 Markov and semi-Markov decision processes
90C15 Stochastic programming
90C39 Dynamic programming
90-01 Introductory exposition (textbooks, tutorial papers, etc.) pertaining to operations research and mathematical programming
90C90 Applications of mathematical programming
90B35 Deterministic scheduling theory in operations research
60G40 Stopping times; optimal stopping problems; gambling theory