×

Average cost optimal policies for Markov control processes with Borel state space and unbounded costs. (English) Zbl 0723.93080

Summary: We show the existence of average cost optimal stationary policies for Markov control processes with Borel state space and unbounded costs per stage, under a set of assumpttions recently introduced by L. I. Sennott [Oper. Res. 37, No.4, 626-633 (1989; Zbl 0675.90091)] for control processes with countable state space and finite control sets.

MSC:

93E20 Optimal stochastic control
60J10 Markov chains (discrete-time Markov processes on discrete state spaces)
90C40 Markov and semi-Markov decision processes

Citations:

Zbl 0675.90091
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Bertsekas, D. P., Dynamic Programming: Deterministic and Stochastic Models (1987), Prentice-Hall: Prentice-Hall Englewood Cliffs, NJ · Zbl 0649.93001
[2] Bertsekas, D. P.; Shreve, S. E., Stochastic Optimal Control: The Discrete-Time Case (1978), Academic Press: Academic Press New York · Zbl 0471.93002
[3] Bhattacharya, R. N.; Majumdar, M., Controlled semi-Markov models - The discounted case, J. Statist. Plann. Inference, 21, 365-381 (1989) · Zbl 0673.93089
[4] Bhattacharya, R. N.; Majumdar, M., Controlled semi-markov models under long-run average rewards, J. Statist. Plann. Inference, 22, 223-242 (1989) · Zbl 0683.49003
[5] Cavazos-Cadena, R., Weak conditions for the existence of optimal stationary policies in average Markov decision chains with unbounded costs, Kybernetika (Prague), 25, 145-156 (1989) · Zbl 0673.90092
[6] R. Cavazos-Cadena and L. Sennott, Comparing recent assumptions for the existence of average optimal stationary policies, Preprint.; R. Cavazos-Cadena and L. Sennott, Comparing recent assumptions for the existence of average optimal stationary policies, Preprint. · Zbl 0763.90092
[7] Georgin, J. P., Contrôle de chaînes de Markov sur des espaces arbitraires, Ann. Inst. H. Poincaré B, 14, 255-277 (1978) · Zbl 0391.60066
[8] Hernández-Lerma, O., Adaptive Markov Control Processes (1989), Springer-Verlag: Springer-Verlag New York · Zbl 0698.90053
[9] Hernández-Lerma, O.; Lasserre, J. B., Value iteration and rolling plans for Markov control processes with unbounded rewards, (LAAS-Report 90037 (1990), LAAS-CNRS: LAAS-CNRS Toulouse, France) · Zbl 0781.90093
[10] Hernández-Lerma, O.; Montes-de-Oca, R.; Cavazos-Cadena, R., Recurrence conditions for Markov decision processes with Borel state space: A survey (1990), Preprint · Zbl 0717.90087
[11] Himmelberg, C. J.; Parthasarathy, T.; van Vleck, F. S., Optimal plans for dynamic programming problems, Math. Oper. Res., 1, 390-394 (1976) · Zbl 0368.90134
[12] Hinderer, K., Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter, (Lecture Notes in Operations Research No. 33 (1970), Springer-Verlag: Springer-Verlag New York) · Zbl 0202.18401
[13] Hordjik, A.; Tijms, H. C., A counterexample in discounted dynamic programming, J. Math. Anal. Appl., 39, 455-457 (1972) · Zbl 0238.49017
[14] Kurano, M., The existence of a minimum pair of state and policy for Markov decision processes under the hypothesis of Doeblin, SIAM J. Control Optim., 27, 296-307 (1989) · Zbl 0677.90085
[15] Kurano, M., Semi-Markov decision processes with a reachable state-subset, Optimization, 20, 305-315 (1989) · Zbl 0677.90087
[16] Lippman, S. A., On dynamic programming with unbounded rewards, Management Sci., 21, 1225-1233 (1975) · Zbl 0309.90017
[17] Piunovski, A. B., General control models with the infinite horizon, Problems Control Inform. Theory, 18, 169-182 (1989) · Zbl 0685.49017
[18] Schäl, M., Conditions for optimality in dynamic programming and for the limit of \(n\)-stage optimal policies to be optimal, Z. Wahrsch. Verw. Geb., 32, 179-196 (1975) · Zbl 0316.90080
[19] Sennott, L. I., A new condition for the existence of optimal stationary policies in average cost Markov decision processes, Oper. Res. Lett., 5, 17-23 (1986) · Zbl 0593.90083
[20] Sennott, L. I., A new condition for the existence of optimum stationary policies in average cost Markov decision processes - unbounded case, (Proc. 25th IEEE Conf. Decision Control. Proc. 25th IEEE Conf. Decision Control, Athens, Greece (Dec. 1986)), 1719-1721
[21] Sennott, L. I., Average cost optimal stationary policies in infinite state Markov decision processes with unbounded costs, Oper. Res., 37, 626-633 (1989) · Zbl 0675.90091
[22] Wakuta, K., Arbitrary state semi-Markov decision processes with unbounded rewards, Optimization, 18, 447-454 (1987) · Zbl 0631.90083
[23] Wijngaard, J., Stationary markovian decision problems and perturbation theory of quasi-compact linear operators, Math. Oper. Res., 2, 91-102 (1977) · Zbl 0395.90081
[24] R. Cavazos-Cadena, A counterexample on the optimality equation in Markov decision chains with the average cost criterion, Report Interno No. 04/2-06/2-09, Depto. de Estadistica y Cálculo, UAAAN, Saltillo, Coahuila, México.; R. Cavazos-Cadena, A counterexample on the optimality equation in Markov decision chains with the average cost criterion, Report Interno No. 04/2-06/2-09, Depto. de Estadistica y Cálculo, UAAAN, Saltillo, Coahuila, México. · Zbl 0738.90082
[25] R.K. Ritt and L.I. Sennott, Optimal stationary policies in general state space Markov decision chains with finite action sets, Preprint.; R.K. Ritt and L.I. Sennott, Optimal stationary policies in general state space Markov decision chains with finite action sets, Preprint. · Zbl 0772.90081
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.