Numerical methods for stochastic control problems in continuous time. (English) Zbl 0754.65068

Kushner, Harold J.; Dupuis, Paul G.

Applications of Mathematics. 24. New York etc.: Springer-Verlag. ix, 439 p. (1992).

The book contains 14 chapters: 1. Review of continuous time models. 2. Controlled Markov chains. 3. Dynamic programming equations. 4. The Markov chain approximation method: Introduction. 5. Construction of the approximating Markov chain. 6. Computational methods for controlled Markov chains. 7. The ergodic cost problems: Formulations and algorithms. 8. Heavy traffic and singular control problems: Examples and Markov chain approximations. 9. Weak convergence and the characterization of processes. 10. Convergence proofs. 11. Convergence for reflecting boundaries, singular control and ergodic cost problems. 12. Finite time problems and nonlinear filtering. 13. Problems from the calculus of variations. 14. The viscosity solution approach to proving convergence of numerical schemes.
The essential idea consists in the approximation of time continuous control problems — driven by Ito equations — by sequences of controlled Markov chains. The controlled Markov chains are constructed by a discretization of controlled Ito equations. The convergence is proved. The book also contains many examples and numerical methods (especially chapter 6).

Reviewer: W.Grecksch (Merseburg)

MSC:

65K10	Numerical optimization and variational techniques
65-02	Research exposition (monographs, survey articles) pertaining to numerical analysis
93E20	Optimal stochastic control
65C99	Probabilistic methods, stochastic differential equations
60F17	Functional limit theorems; invariance principles

Keywords:

monograph; continuous time models; Controlled Markov chains; Dynamic programming; ergodic cost problems; algorithms; Heavy traffic; singular control; Weak convergence; characterization of processes; reflecting boundaries