On some limit properties of the reward from a Markov replacement process. (English) Zbl 0542.60089

Kunderová, Pavla

Acta Univ. Palacki. Olomuc., Fac. Rerum Nat. 76, Math. 22, 143-156 (1983).

The finite state Markov processes with replacements are given in connection with the point processes. The marked point process with rewards is considered with marks \(\ell =1,...,q\) such that \(\ell \sim(i,k)\) means the transition from state i into state k, \(\ell \sim(i,+k)\) means the replacement (the instantaneous shift of the trajectory from state i into state k) and \(\ell \sim(i,k,+k')\) means the transition \(i\to k\) and in the same moment the replacement \(k\to k'.\)
A compensator (integral from the intensity) \(\{\) \(\tilde A_ t,t\geq 0\}=\{^ 1\tilde A_ t,...,^ q\tilde A_ t\}\) of the process is investigated if the process is under such a stationary replacement policy of destination f, under which one class of recurrent states exists only. Next there is considered the compensator \(\{A_ t,t\geq 0\}\) of the process under a common replacement policy F. Let \(R_ t\) be the reward from the process up to the time t and \(\theta\) the mean reward per time unit.
Theorem 1: If \(\lim_{t\to \infty}(^{\ell}A_ t-^{\ell}\tilde A_ t)=0\) F-almost sure holds for all \(\ell =1,...,q\) then \(\lim_{t\to \infty}t^{-1}R_ t=\theta\) F-almost sure. In Theorem 2 there are proved conditions under which \(t^{-1/2}(R_ t-\theta t)\) for \(t\to \infty\) has asymptotically normal distribution N(0,\(\xi)\), where \(\xi\) is a certain constant.

MSC:

60K20	Applications of Markov renewal processes (reliability, queueing networks, etc.)
60G55	Point processes (e.g., Poisson, Cox, Hawkes processes)
60J20	Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.)

Keywords:

marked point process; replacement policy

References:

[1]	Kunderová V.: On a mean reward from Markov replacement process with only one isolated class of recurrent states. Acta UP Olomucensis, F. R. N. 1979, Tom. 61.
[2]	Kunderová P.: On limit properties of the reward from a Markov replacement process. Acta UP Olomucensis, F. R. N. 1981, Tom 69. · Zbl 0484.60071
[3]	Liptser R. S., Shiryayev A. N.: Statistics of Random Processes II. New York, Springer Verlag, 1978. · Zbl 0369.60001
[4]	Loève M.: Probability Theory. Princeton · Zbl 0095.12201
[5]	Mandl P.: Some results on Markovian replacement process. J. Appl. Prob. 8, 357-365 (1971). · Zbl 0222.60058
[6]	Mandl P.: An identity for Markovian replacement process. J. Appl. Prob. 6, 348-354 (1969). · Zbl 0192.55203