Approximate uniformization for continuous-time Markov chains with an application to performability analysis. (English) Zbl 0753.60066

van Dijk, Nico M.

Stochastic Processes Appl. 40, No. 2, 339-357 (1992).

Author’s summary: An approximate version of the standard uniformization technique is introduced for application to continuous-time Markov chains with unbounded jump rates. This technique is shown to be asymptotically exact and an error bound for the order of its accuracy is provided. An illustrative queueing application is included.

Reviewer: W.Schenk (Dresden)

MSC:

60J27	Continuous-time Markov processes on discrete state spaces
90B18	Communication networks in operations research
90C40	Markov and semi-Markov decision processes
60K10	Applications of renewal theory (reliability, demand theory, etc.)

Keywords:

uniformization technique; continuous-time Markov chains; jump rates; queueing application

References:

[1]	Gihman, I. I.; Skorohod, A. V., Controlled Stochastic Processes (1979), Springer: Springer Berlin · Zbl 0404.60061
[2]	Jensen, A., Markov chains as an aid in the study of Markov processes, Skand. Aktuartidskr., 36, 87-91 (1953) · Zbl 0051.35607
[3]	Kemeny, J. G.; Snell, J. L.; Knapp, A. W., Denumerable Markov Chains (1966), Van Nostrand: Van Nostrand Princeton, NJ · Zbl 0149.13301
[4]	Jaiswal, N. K., Priority Queues (1968), Academic Press: Academic Press New York · Zbl 0179.47904
[5]	Pliska, S. R., Controlled jump processes, Stochastic Process. Appl., 3, 259-282 (1975) · Zbl 0313.60055
[6]	Ross, S. M., Approximating transition probabilities and mean occupation times in continuous-time Markov chains, Probab. Eng. Inf. Sci., 1, 164-251 (1987) · Zbl 1133.60344
[7]	Seneta, E., Non-negative Matrices and Markov Chains (1980), Springer: Springer Berlin · Zbl 1099.60004
[8]	Serfozo, R. F., An equivalence between continuous and discrete time Markov decision processes, Oper. Res., 27, 616-620 (1979) · Zbl 0413.90079
[9]	Shanthikumar, J. G.; Yao, D. D., General queueing networks: Representation and stochastic monotonicity, (Res. Rep. (1987), Univ. of California: Univ. of California Berkeley, CA) · Zbl 0686.60099
[10]	Shanthikumar, J. G.; Yao, D. D., Stochastic monotonicity of the queue lengths in closed queueing networks, Oper. Res., 35, 583-588 (1987) · Zbl 0652.60101
[11]	Stoyan, D., Comparison Methods for Queues and other Stochastic Models (1983), Wiley: Wiley New York
[12]	Suri, R., A concept of monotonicity and its characterization for closed queueing networks, Oper. Res., 33, 606-624 (1985) · Zbl 0567.90040
[13]	Tijms, H. C., Stochastic Modelling and Analysis ; A Computational Approach Analysis (1986), Wiley: Wiley New York
[14]	Tijms, H. C.; Eikeboom, A. M., A simple technique in Markovian control with applications to resource allocation, Oper. Res. Lett., 1, 25-32 (1986) · Zbl 0606.90128
[15]	Van Dijk, N. M., On the finite horizon Bellman equation for controlled Markov jump models with unbounded characteristics, Stochastic Process. Appl., 28, 141-157 (1988) · Zbl 0645.93072
[16]	Van Dijk, N. M., Simple bounds for queueing systems with breakdowns, Performance Evaluation, 8, 117-128 (1988) · Zbl 0698.90034
[17]	Van Dijk, N. M., Truncation of Markov chains with applications to queueing, Oper. Res. (1991), to appear in: · Zbl 0749.60069
[18]	Van Dijk, N. M.; Van der Wal, J., Simple bounds and monotonicity results for multi-server exponential tandem queues, Queueing Syst., 4, 1-19 (1989) · Zbl 0664.60093
[19]	Whitt, W., Comparing counting processes and queues, Adv. Appl. Probab., 13, 207-220 (1981) · Zbl 0449.60064
[20]	Yoon, B. S.; Shanthikumar, B. S., Bounds and approximations for the transient behaviour of continuous time Markov chains, Probab. Eng. Inform. Sci., 3, 175-198 (1989) · Zbl 1134.60366