zbMATH — the first resource for mathematics

On the Markov renewal theorem. (English) Zbl 0789.60066
Summary: Let \((S,{\mathcal S})\) be a measurable space with countably generated \(\sigma\)-field \({\mathcal S}\) and \((M_ n,X_ n)_{n \geq 0}\) a Markov chain with state space \(S \times \mathbb{R}\) and transition kernel \(\mathbb{P}:S \times ({\mathcal S} \otimes {\mathcal B}) \to[0,1]\). Then \((M_ n,S_ n)_{n \geq 0}\), where \(S_ n=X_ 0+\cdots+X_ n\) for \(n \geq 0\), is called the associated Markov random walk. Markov renewal theory deals with the asymptotic behavior of suitable functionals of \((M_ n,S_ n)_{n \geq 0}\) like the Markov renewal measure \(\sum_{n \geq 0} P((M_ n,S_ n) \in A \times(t+B))\) as \(t \to \infty\) where \(A \in {\mathcal S}\) and \(B\) denotes a Borel subset of \(\mathbb{R}\). It is shown that the Markov renewal theorem as well as a related ergodic theorem for semi-Markov processes hold true if only Harris recurrence of \((M_ n)_{n \geq 0}\) is assumed. This was proved by purely analytical methods by V. M. Shurenkov [Theory Probab. Appl. 29, 247-265 (1985); translation from Teor. Veroyatn. Primen. 29, No. 2, 248-263 (1984; Zbl 0544.60082)] in the one- sided case where \(\mathbb{P}(x,S \times [0, \infty))=1\) for all \(x \in S\). Our proof uses probabilistic arguments, notably the construction of regeneration epochs for \((M_ n)_{n \geq 0}\) such that \((M_ n,X_ n)_{n \geq 0}\) is at least nearly regenerative and an extension of Blackwell’s renewal theorem to certain random walks with stationary, 1- dependent increments.

60K15 Markov renewal processes, semi-Markov processes
60G50 Sums of independent random variables; random walks
60K05 Renewal theory
60J05 Discrete-time Markov processes on general state spaces
Full Text: DOI
[1] Alsmeyer, G., Random walks with stochastically bounded increments: renewal theory, (1992), submitted for publication
[2] Alsmeyer, G., Random walks with stochastically bounded increments: renewal theory via Fourier analysis, (1992), submitted for publication
[3] Asmussen, S., Applied probability and queues, (1987), Wiley New York · Zbl 0624.60098
[4] Athreya, K.B.; Ney, P., A new approach to the limit theory of recurrent Markov chains, Trans. amer. math. soc., 245, 493-501, (1978) · Zbl 0397.60053
[5] Athreya, K.B.; Ney, P., Limit theorems for semi-Markov processes, Bull. austral. math. soc., 19, 283-294, (1978) · Zbl 0392.60068
[6] Athreya, K.B.; McDonald, D.; Ney, P., Limit theorems for semi-Markov processes and renewal theory for Markov chains, Ann. probab., 6, 788-797, (1978) · Zbl 0397.60052
[7] Berbee, H., Random walks with stationary increments and renewal theory, () · Zbl 0443.60083
[8] Jacod, J., Théorème de renouvellement et classification pour LES chaines semi-markoviennes, Ann. inst. H. Poincaré B, 7, 355-387, (1971) · Zbl 0217.50502
[9] Kesten, H., Renewal theory for functionals of a Markov chain with general state space, Ann. probab., 2, 355-387, (1974) · Zbl 0303.60090
[10] Lalley, S., A renewal theorem for a class of stationary sequences, Probab. theory rel. fields, 72, 195-213, (1986) · Zbl 0597.60083
[11] Ney, P.; Nummelin, E.; Janssen, J., Some limit theorems for Markov additive processes, Semi-Markov models: theory and applications, 3-12, (1986)
[12] Nummelin, E., A splitting technique for harris recurrent Markov chains, Z. wahrsch. verw. gebiete, 43, 309-318, (1978) · Zbl 0364.60104
[13] Orey, S., Change of time scale for Markov processes, Trans. amer. math. soc., 99, 384-390, (1961) · Zbl 0102.14003
[14] Pitman, J.; Speed, T., A note on random times, Stochastic process. appl., 1, 369-374, (1973) · Zbl 0275.60060
[15] Shurenkov, V.M., On the theory of Markov renewal, Theory probab. appl., 29, 247-265, (1984) · Zbl 0557.60078
[16] Thorisson, H., Construction of a stationary regenerative process, Stochastic process. appl., 42, 237-253, (1992) · Zbl 0765.60024
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.