×
Cavazos-Cadena, Rolando

Finite-state approximations for denumerable state discounted Markov decision processes. (English) Zbl 0606.90132

Appl. Math. Optimization 14, 1-26 (1986).
The paper generalizes the finite-state interactive scheme of D. J. White [in: Recent developments in Markov decision processes (1980; Zbl 0547.90064), and J. Math. Anal. Appl. 86, 292-306 (1982; Zbl 0533.90094)] to more general (denumerable) state sets and more general conditions of convergence. The rate of convergence is studied intensively and the asymptotic discount optimality of the policies generated by the algorithm is proved.
Reviewer: G.Hübner

Cited in 10 Documents

MSC:

90C40 Markov and semi-Markov decision processes

Keywords:

denumerable state discounted Markov decision processes; successive approximations; finite-state approximations; unbounded rewards; general conditions of convergence; rate of convergence; asymptotic discount optimality

Citations:

Zbl 0547.90064; Zbl 0533.90094
Cite Review PDF
Full Text: DOI

References:

[1] Fox BL (1971) Finite-state approximations for denumerable-state dynamic programs. J Math Anal Appl 34:665-670 · Zbl 0217.28403 · doi:10.1016/0022-247X(71)90106-5
[2] Harrison JM (1972) Discrete dynamic programming with unbounded rewards. Ann Math Statist 43:636-644 · Zbl 0262.90064 · doi:10.1214/aoms/1177692643
[3] Hern?ndez-Lerma O (1984) Finite-state approximations for denumerable multidimensional discounted Markov decision processes. J Math Anal Appl (to appear)
[4] Hern?ndez-Lerma O, Marcus, SI (1985) Adaptive control of discounted Markov decision chains. J Optim Theory Appl 46:227-235 · Zbl 0543.90093 · doi:10.1007/BF00938426
[5] Lippman SA (1975) On dynamic programming with unbounded rewards. Management Sci 21:1225-1233 · Zbl 0309.90017 · doi:10.1287/mnsc.21.11.1225
[6] Ross SM (1976) Applied Probability Models with Optimization Applications. Holden-Day, San Francisco
[7] Sch?l M (1981) Estimation and control in discounted stochastic dynamic programming. Preprint No. 428, Inst Angew Math, Univ of Bonn
[8] Wessels J (1977) Markov programming by successive approximations with respect to weighted supremum norms. J Math Anal Appl 58:326-335 · Zbl 0354.90087 · doi:10.1016/0022-247X(77)90210-4
[9] White DJ (1980) Finite state approximations for denumerable state infinite horizon discounted Markov decision processes: the method of successive approximations. In: Hartley R, Thomas LC, White DJ (eds) Recent Developments in Markov Decision Processes. Academic Press, New York · Zbl 0428.90082
[10] White DJ (1982) Finite state approximations for denumerable state infinite horizon discounted Markov decision processes with unbounded rewards. J Math Anal Appl 86:292-306 · Zbl 0533.90094 · doi:10.1016/0022-247X(82)90271-2
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.