Finite Markov chain models skip-free in one direction.(English)Zbl 0551.90097

Finite Markov processes are considered, with bidimensional state space, such that transitions from state (n,i) to state (m,j) are possible only if $$m\leq n+1$$. The analysis leads to efficient computational algorithms, to determine the stationary probability distribution, and moments of first passage times.

MSC:

 90C40 Markov and semi-Markov decision processes 60K15 Markov renewal processes, semi-Markov processes 90B22 Queues and service in operations research
Full Text:

References:

 [1] Carroll, Operations Research 30 pp 490– (1982) [2] , and , ”Finite birth-and-death models in randomly changing environments,” Journal of Applied Probability, in 1985. [3] Hajek, Journal of Applied Probability 19 pp 488– (1982) [4] , and , ”Row-continuous finite Markov chains, structure and algorithms,” Technical Report No. 8115. Graduate School of Management, University of Rochester, 1981. [5] and , Finite Markov Chains, Van Nostrand, Princeton, NJ, 1960. [6] and , ”Numerical methods for a class of Markov chains arising in queueing theory,” Technical Report 78/10, Applied Mathematics Institute, University of Delaware, Newark, 1978. [7] Neuts, Operations Research 27 pp 767– (1979) [8] Neuts, Journal of Applied Probability 17 pp 291– (1980) [9] Matrix-Geometric Solutions in Stochastic Models–An Algorithmic Approach, The Johns Hopkins University Press, Baltimore, 1981. · Zbl 0469.60002 [10] Neuts, Operations Research 30 pp 480– (1982) [11] Neuts, OR Spektrum 2 pp 227– (1981) [12] Torrez, Journal of Applied Probability 16 pp 709– (1979) · Zbl 0423.92020 [13] Wikarski, Elektronische Informationsverarbeitung und Kybernetik 16 pp 615– (1980) [14] and , Handbook for Automatic Computation, Volume II, Linear Algebra, Springer-Verlag, Berlin, 1971.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.