×

zbMATH — the first resource for mathematics

Risk sensitive control of Markov processes in countable state space. (English) Zbl 0866.93101
Summary: We consider infinite horizon risk-sensitive control of Markov processes with discrete-time and denumerable state space. This problem is solved by proving, under suitable conditions, that there exists a bounded solution to the dynamic programming equation. The dynamic programming equation is transformed into an Isaacs equation for a stochastic game, and the vanishing discount method is used to study its solution. In addition, we prove that the existence conditions are also necessary.

MSC:
93E20 Optimal stochastic control
91A60 Probabilistic games; gambling
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Arapostathis, A.; Borkar, V.S.; Fernandez-Gaucherand, E.; Ghosh, M.K.; Marcus, S.I., Discrete-time controlled Markov processes with average cost criterion: a survey, SIAM J. control optim., 31, 282-344, (1993) · Zbl 0770.93064
[2] J.S. Baras and M.R. James, Robust and risk-sensitive output feedback control for finite state machines and hidden Markov models, to appear in: J. Math. Systems Estim. Control. · Zbl 0911.93055
[3] Bensoussan, A.; Van Schuppen, J.H., Optimal control of partially observable stochastic systems with exponential of integral performance index, SIAM J control optim., 23, 599-613, (1985) · Zbl 0574.93067
[4] Cavazos-Cadena, R., Necessary conditions for the optimality equation in average reward Markov decision processes, Appl. math. optim., 19, 97-112, (1989) · Zbl 0663.90094
[5] Chung, K.J.; Sobel, M.J., Discounted MDP’s: distribution functions and exponential utility maximization, SIAM J. control optim., 25, 49-62, (1987) · Zbl 0617.90085
[6] P. Dai Pra, L. Meneghini and W.J. Runggaldier, Some connections between stochastic control and dynamic games, preprint. · Zbl 0874.93096
[7] P. Dupuis and R.S. Ellis, A Weak Convergence Approach to the Theory of Large Deviations, forthcoming (Wiley, New York). · Zbl 0904.60001
[8] Eagle, J.N., A utility criterion for the Markov decision process, ()
[9] Fernández-Gaucherand, E.; Arapostathis, A.; Marcus, S.I., Remarks on the existence of solutions to the average cost optimality equation in Markov decision processes, Systems control lett., 15, 425-432, (1990) · Zbl 0742.93079
[10] Fernández-Gaucherand, E.; Marcus, S.I., Risk-sensitive optimal control of hidden Markov models: a case study, (), 1657-1662
[11] W.H. Fleming and D. Hernandez-Hernandez, Risk-sensitive control of finite state machines on an infinite horizon, to appear in: SIAM J. Control Optim. · Zbl 0891.93085
[12] Fleming, W.H.; McEneaney, W.M., Risk-sensitive control and differential games, (), 185-197 · Zbl 0788.90097
[13] Fleming, W.H.; McEneaney, W.M., Risk-sensitive control on an infinite horizon, SIAM J. control optim., 33, 1881-1915, (1995) · Zbl 0949.93079
[14] Hernández-Lerma, O., Adaptive Markov control processes, (1989), Springer New York · Zbl 0698.90053
[15] Howard, R.A.; Matheson, J.E., Risk-sensitive Markov decision processes, Management sci., 18, 356-369, (1972) · Zbl 0238.90007
[16] Jacobson, D.H., Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games, IEEE trans. automat. control, 18, 124-131, (1973) · Zbl 0274.93067
[17] James, M.R., Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games, Math. control signals systems, 5, 401-417, (1992) · Zbl 0761.93076
[18] James, M.R.; Baras, J.S.; Elliott, R.J., Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems, IEEE trans. automat. control, 39, 780-792, (1994) · Zbl 0807.93067
[19] Runolfsson, T., The equivalence between infinite horizon control of stochastic systems with exponential-of-integral performance index and stochastic differential games, IEEE trans. automat. control, 39, 1551-1563, (1994) · Zbl 0930.93084
[20] Whittle, P., Risk-sensitive optimal control, (1990), Wiley New York · Zbl 0718.93068
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.