# zbMATH — the first resource for mathematics

Time-discretization for controlled Markov processes. I: General approximation results. (English) Zbl 0874.93094
[Part II, ibid. 32, No. 2, 139-158 (1996), is reviewed below.]
The authors prove a convergence theorem for approximation of a continuous-time controlled Markov process by a discrete-time controlled Markov process. The theorem is analogous to the well-known Lax-Richtmeyer theorem which states that a consistent and stable difference method converges.
##### MSC:
 93E20 Optimal stochastic control 93C57 Sampled-data control/observation systems
Full Text:
##### References:
 [1] A. Bensoussan M. Robin: On the convergence of the discrete time dynamic programming equation for general semi-groups. SIAM J. Control Optim. 20 (1982), 1, 722-746. · Zbl 0488.93062 [2] N. Christopeit: Discrete approximation of continuous time stochastic control systems. SIAM J. Control Optim. 21 (1983), 1, 17-40. · Zbl 0508.93065 [3] B. T. Doshi: Optimal control of the service rate in an $$M|G|1$$-queueing system. Adv. in Appl. Probab. 10 (1978), 682-701. · Zbl 0381.60086 [4] W. H. Fleming R. W. Rishel: Deterministic and Stochastic Optimal Control. Springer Verlag, Berlin 1975. · Zbl 0323.49001 [5] I. I. Gihman A. V. Skorohod: Controlled Stochastic Processes. Springer Verlag, Berlin 1979. · Zbl 0404.60061 [6] U. G. Haussmann: A discrete approximation to optimal stochastic. Analysis and Optimization of Stochastic Systems, Academic Press, London 1980, pp. 229-241. · Zbl 0476.93082 [7] A. Hordijk F. A. Van der Duyn Schouten: Average optimal policies in Markov decision drift processes with applications to queueing and replacement model. Adv. in Appl. Probab. 15 (1983), 274-303. · Zbl 0517.90091 [8] A. Hordijk F. A. Van der Duyn Schouten: Discretization and weak convergence in Markov decision drift processes. Math. Oper. Res. 9 (1984), 1, 112-141. · Zbl 0531.90097 [9] A. Hordijk F. A. Van der Duyn Schouten: Markov decision drift processes; Conditions for optimality obtained by discretization. Math. Oper. Res. 10 (1985), 160-173. · Zbl 0573.90099 [10] A. Hordijk F. A. Van der Duyn Schouten: On the optimality of $$(s,S)$$-policies in continuous review inventory models. SIAM J. Appl. Math. 46 (1986), 912-929. · Zbl 0648.90021 [11] G. M. Koole: Stochastic Scheduling and Dynamic Programming. Ph.D. Thesis, University of Leiden 1992. · Zbl 0851.90065 [12] T. G. Kurtz: Extensions of Trotter’s operator semigroup approximations theorems. J. Funct. Anal. 3 (1969), 111-132. · Zbl 0174.18401 [13] H. J. Kushner: Probability Methods for Approximation in Stochastic Control and for Elliptic Equations. Academic Press, New York 1977. · Zbl 0547.93076 [14] P. D. Lax R. D. Richtmeyer: Survey of the stability of linear finite difference equations. Comm. Pure Appl. Math. 9 (1956), 267-293. · Zbl 0072.08903 [15] T. Meis U. Marcowitz: Numerical Solution of Partial Differential Equations. Springer Verlag, Berlin 1981. · Zbl 0446.65045 [16] H. J. Plum: Impulsive and continuously acting control of jump processes – Time discretization. Stochastics and Stochastic Reports 36 (1991), 163-192. · Zbl 0739.60076 [17] R. Rishel: Necessary and sufficient dynamic programming conditions for continuous time stochastic optimal control. SIAM J. Control 8 (1970), 4, 559-571. · Zbl 0206.45804 [18] R. Rishel: Controls optimal from the toward and dynamic programming for systems of controlled jump processes. Math. Programming Study 6 (1976), 125-153. [19] F. A. Van der Duyn Schouten: Markov Decision Processes with Continuous Time Parameter. Mathematical Centre Tract 164, Amsterdam 1983. · Zbl 0519.90052 [20] N. M. Van Dijk: Controlled Markov Processes; Time Discretization/Networks of Queues. Ph.D. Thesis, University of Leiden 1983. [21] N. M. Van Dijk: On the finite horizon Bellman equation for controlled Markov jump models with unbounded characteristics: existence and approximations. Stochastic Process. Appl. 28 (1988), 141-157. · Zbl 0645.93072
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.