Markov decision processes with state-dependent discount factors and unbounded rewards/costs. (English) Zbl 1235.90178

Summary: This paper deals with discrete-time Markov decision processes with state-dependent discount factors and unbounded rewards/costs. Under general conditions, we develop an iteration algorithm for computing the optimal value function, and also prove the existence of optimal stationary policies. Furthermore, we illustrate our results with a cash-balance model.


90C40 Markov and semi-Markov decision processes
Full Text: DOI


[1] Altman, E., Constrained Markov decision processes, (1999), Chapman and Hall, CRC Press London · Zbl 0963.90068
[2] Berument, H.; Kilinc, Z.; Ozlale, U., The effects of different inflation risk prepius on interest rate spreads, Physica A, 333, 317-324, (2004)
[3] Borkar, V.S., A convex analytic approach to Markov decision processes, Probab. theory related fields, 78, 583-602, (1988) · Zbl 0628.90090
[4] Carmon, Y.; Shwartz, A., Markov decision processes with exponentially representable discounting, Oper. res. lett., 37, 51-55, (2009) · Zbl 1154.90610
[5] Feinberg, E.A.; Shwartz, A., Markov decision models with weighted discounted criteria, Math. oper. res., 19, 152-168, (1994) · Zbl 0803.90123
[6] González-Hernández, J.; López-Martínez, R.R.; Pérez-Hernández, J.R., Markov control processes with randomized discounted cost, Math. methods oper. res., 65, 27-44, (2007) · Zbl 1126.90075
[7] Hernández-Lerma, O.; Lasserre, J.B., Discounted cost Markov decision processes on Borel spaces: the linear programming formulation, J. math. anal. appl., 183, 335-351, (1994) · Zbl 0820.90124
[8] Hernández-Lerma, O.; Lasserre, J.B., Discrete-time Markov control processes: basic optimality criteria, (1996), Springer-Verlag New York
[9] Hernández-Lerma, O.; Lasserre, J.B., Further topics on discrete-time Markov control processes, (1999), Springer-Verlag New York · Zbl 0928.93002
[10] Hinderer, K., ()
[11] Hordjik, A.; Yushkevich, A.A., Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards, Math. methods oper. res., 50, 421-448, (1999) · Zbl 0939.90020
[12] Howard, R.A., Dynamic programming and Markov processes, (1960), Wiley New York · Zbl 0091.16001
[13] Newell, R.G.; Pizer, W.A., Discounting the distant future: how much do uncertain rates increase valuation?, J. environ. econ. manag., 46, 52-71, (2003) · Zbl 1041.91502
[14] Piunovskiy, A.B., Optimal control of random sequences in problems with constraits, (1997), Kluwer Dordrecht · Zbl 0894.93001
[15] Puterman, M.L., Markov decision processes: discrete stochastic dynamic programming, (1994), Wiley New York · Zbl 0829.90134
[16] Schäl, M., Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal, Z. wahrscheinlichkeitstheor. verwandte geb., 32, 179-196, (1975) · Zbl 0316.90080
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.