##
**On some limit properties of the reward from a Markov replacement process.**
*(English)*
Zbl 0542.60089

The finite state Markov processes with replacements are given in connection with the point processes. The marked point process with rewards is considered with marks \(\ell =1,...,q\) such that \(\ell \sim(i,k)\) means the transition from state i into state k, \(\ell \sim(i,+k)\) means the replacement (the instantaneous shift of the trajectory from state i into state k) and \(\ell \sim(i,k,+k')\) means the transition \(i\to k\) and in the same moment the replacement \(k\to k'.\)

A compensator (integral from the intensity) \(\{\) \(\tilde A_ t,t\geq 0\}=\{^ 1\tilde A_ t,...,^ q\tilde A_ t\}\) of the process is investigated if the process is under such a stationary replacement policy of destination f, under which one class of recurrent states exists only. Next there is considered the compensator \(\{A_ t,t\geq 0\}\) of the process under a common replacement policy F. Let \(R_ t\) be the reward from the process up to the time t and \(\theta\) the mean reward per time unit.

Theorem 1: If \(\lim_{t\to \infty}(^{\ell}A_ t-^{\ell}\tilde A_ t)=0\) F-almost sure holds for all \(\ell =1,...,q\) then \(\lim_{t\to \infty}t^{-1}R_ t=\theta\) F-almost sure. In Theorem 2 there are proved conditions under which \(t^{-1/2}(R_ t-\theta t)\) for \(t\to \infty\) has asymptotically normal distribution N(0,\(\xi)\), where \(\xi\) is a certain constant.

A compensator (integral from the intensity) \(\{\) \(\tilde A_ t,t\geq 0\}=\{^ 1\tilde A_ t,...,^ q\tilde A_ t\}\) of the process is investigated if the process is under such a stationary replacement policy of destination f, under which one class of recurrent states exists only. Next there is considered the compensator \(\{A_ t,t\geq 0\}\) of the process under a common replacement policy F. Let \(R_ t\) be the reward from the process up to the time t and \(\theta\) the mean reward per time unit.

Theorem 1: If \(\lim_{t\to \infty}(^{\ell}A_ t-^{\ell}\tilde A_ t)=0\) F-almost sure holds for all \(\ell =1,...,q\) then \(\lim_{t\to \infty}t^{-1}R_ t=\theta\) F-almost sure. In Theorem 2 there are proved conditions under which \(t^{-1/2}(R_ t-\theta t)\) for \(t\to \infty\) has asymptotically normal distribution N(0,\(\xi)\), where \(\xi\) is a certain constant.

### MSC:

60K20 | Applications of Markov renewal processes (reliability, queueing networks, etc.) |

60G55 | Point processes (e.g., Poisson, Cox, Hawkes processes) |

60J20 | Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) |

### References:

[1] | Kunderová V.: On a mean reward from Markov replacement process with only one isolated class of recurrent states. Acta UP Olomucensis, F. R. N. 1979, Tom. 61. |

[2] | Kunderová P.: On limit properties of the reward from a Markov replacement process. Acta UP Olomucensis, F. R. N. 1981, Tom 69. · Zbl 0484.60071 |

[3] | Liptser R. S., Shiryayev A. N.: Statistics of Random Processes II. New York, Springer Verlag, 1978. · Zbl 0369.60001 |

[4] | Loève M.: Probability Theory. Princeton · Zbl 0095.12201 |

[5] | Mandl P.: Some results on Markovian replacement process. J. Appl. Prob. 8, 357-365 (1971). · Zbl 0222.60058 · doi:10.2307/3211905 |

[6] | Mandl P.: An identity for Markovian replacement process. J. Appl. Prob. 6, 348-354 (1969). · Zbl 0192.55203 · doi:10.2307/3212005 |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.