zbMATH — the first resource for mathematics

Algorithms for determining the state-time probabilities and the limit matrix in Markov chains. (English) Zbl 1228.93131
Summary: New calculation procedures for finding the probabilities of state transitions of the system in Markov chains based on dynamic programming are developed and polynomial time algorithms for determining the limit state matrix in such processes are proposed. Computational complexity aspects and possible applications of the proposed algorithms for the stochastic optimization problems are characterized.
93E20 Optimal stochastic control
49L20 Dynamic programming in optimal control and differential games
60J10 Markov chains (discrete-time Markov processes on discrete state spaces)
PDF BibTeX Cite
Full Text: Link