On the first passage \(g\)-mean-variance optimality for discounted continuous-time Markov decision processes. (English) Zbl 1322.90108


90C40 Markov and semi-Markov decision processes
93E20 Optimal stochastic control
Full Text: DOI


[1] N. Baüerle and U. Rieder, Markov Decision Processes with Applications to Finance, Springer, Heidelberg, 2011.
[2] E. A. Feinberg, Continuous time discounted jump Markov decision processes: A discrete-event approach, Math. Oper. Res., 29 (2004), pp. 492–524. · Zbl 1082.90126
[3] E. A. Feinberg, Reduction of discounted continuous-time MDPs with unbounded jump and reward rates to discrete-time total-reward MDPs, in Optimization, Control, and Applications of Stochastic Systems, Systems Control Found. Appl., D. Hernández-Hernández and A. Minjarez-Sosa, eds., Birkhäuser/Springer, New York, 2012, pp. 77–97. · Zbl 1374.90402
[4] Ch. P. Fu, A. Lari-Lavassani, and X. Li, Dynamic mean-variance portfolio selection with borrowing constraint, European J. Oper. Res., 200 (2010), pp. 312–319. · Zbl 1183.91192
[5] X. P. Guo, Continuous-time Markov decision processes with discounted rewards: The case of Polish spaces, Math. Oper. Res., 32 (2007), pp. 73–87. · Zbl 1278.90426
[6] X. P. Guo and O. Hernández-Lerma, Continuous-Time Markov Decision Processes, Springer, Heidelberg/Dordrecht/New York, 2009.
[7] X. P. Guo and X. Y. Song, Discounted continuous-time constrained Markov decision processes in Polish spaces, Ann. Appl. Probab., 21 (2011), pp. 2016–2049. · Zbl 1258.90104
[8] X. P. Guo, X. Y. Song, and Y. Zhang, First passage optimality for continuous-time Markov decision processes with varying discount factors and history-dependent policies, IEEE Trans. Automat. Control, 59 (2014), pp. 163–174. · Zbl 1360.90278
[9] X. P. Guo and A. Piunovskiy, Discounted continuous-time Markov decision processes with constraints: Unbounded transition and loss rates, Math. Oper. Res., 36 (2011), pp. 105–132. · Zbl 1218.90209
[10] X. P. Guo, L. E. Ye, and G. Yin, A mean-variance optimization problem for discounted Markov decision processes, European J. Oper. Res., 220 (2012), pp. 423–429. · Zbl 1253.90214
[11] O. Hernández-Lerma and T. E. Govindan, Nonstationary continuous-time Markov control processes with discounted costs on infinite horizon, Acta Appl. Math., 67 (2001), pp. 277–293. · Zbl 1160.93397
[12] O. Hernández-Lerma and J. B. Lasserre, Further Topics on Discrete-Time Markov Control Processes, Springer-Verlag, New York, 1999. · Zbl 0928.93002
[13] O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer-Verlag, New York, 1996.
[14] Q. Y. Hu, Continuous-time Markov decision processes with discounted moment criterion, J. Math. Anal. Appl., 203 (1996), pp. 1–12. · Zbl 0858.90135
[15] S. C. Jaquette, Markov decision processes with a new optimality criterion: Continuous time, Ann. Statist., 3 (1975), pp. 547–553. · Zbl 0321.90051
[16] P. Kakumanu, Continuously discounted Markov decision models with countable state and action space, Ann. Math. Statist., 42 (1971), pp. 919–926. · Zbl 0234.93027
[17] S. A. Lippman, Applying a new device in the optimization of exponential queuing systems, Oper. Res., 23 (1975), pp. 687–710. · Zbl 0312.60048
[18] J. Y. Liu and S. M. Huang, Markov decision processes with distribution function criterion of first-passage time, Appl. Math. Optim., 43 (2001), pp. 187–201. · Zbl 1014.90110
[19] H. M. Markowitz, Portfolio selection, J. Finance, 7 (1952), pp. 77–91.
[20] H. M. Markowitz, Mean-Variance Analysis in Portfolio Choice and Capital Markets, Basil Blackwell, Oxford, UK, 1987. · Zbl 0757.90003
[21] B. L. Miller, Finite state continuous time Markov decision processes with an infinite planning horizon, J. Math. Anal. Appl., 22 (1968), pp. 552–569. · Zbl 0157.50301
[22] A. Piunovskiy and Y. Zhang, Discounted continuous-time Markov decision processes with unbounded rates: The convex analytic approach, SIAM J. Control Optim., 49 (2011), pp. 2032–2061. · Zbl 1242.90283
[23] T. Prieto-Rumeau and O. Hernández-Lerma, Selected Topics on Continuous-Time Controlled Markov Chains and Markov Games, Imperial College Press, London, 2012. · Zbl 1269.60004
[24] M. L. Puterman, Markov Decision Processes, Wiley, New York, 1994.
[25] L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley, New York, 1999. · Zbl 0997.93503
[26] L. E. Ye and X. P. Guo, Continuous-time Markov decision processes with state-dependent discount factors, Acta Appl. Math., 121 (2012), pp. 5–27. · Zbl 1281.90082
[27] G. Yin and X. Y. Zhou, Markowitz’s mean-variance portfolio selection with regime switching: From discrete-time models to their continuous-time limits, IEEE Trans. Automat. Control, 49 (2004), pp. 349–360. · Zbl 1366.91148
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.