×

Cycle representations of Markov processes. (English) Zbl 0824.60002

Applications of Mathematics. 28. Berlin: Springer-Verlag. xv, 194 p. (1995).
Let \(S\) be a denumerable set. A cycle is \(S\) is a finite collection of distinct elements of \(S\) up to cyclic permutations. With a cycle \(\widehat c = (i_ 1, \dots, i_ n)\) in \(S\) the circuit \(c = (i_ 1, \dots, i_ n, i_ 1)\) is associated. If \(\xi = (\xi_ n)_{n \geq 0}\) is an \(S\)-valued recurrent Markov chain, then on any sample path of \(\xi\) a sequence of cycles occurs naturally. The occurrence of a cycle \(\widehat c\) in this sequence is associated with that of the corresponding circuit \(c\) in what remains of the sample path after discarding all the preceding cycles. For instance, the sequence of cycles occurring on the sample path \((i_ 1, i_ 1, i_ 2, i_ 3, i_ 4, i_ 3, i_ 5, i_ 6, i_ 7, i_ 5, i_ 1, \dots)\) is \((i_ 1)\), \((i_ 3, i_ 4)\), \((i_ 5, i_ 6, i_ 7)\), \((i_ 1, i_ 2, i_ 3, i_ 5), \dots\). For a positive-recurrent irreducible \(\xi\) with invariant probability distribution \((\pi_ i)_{i \in S}\) and transition matrix \((p_{ij})_{i,j \in S}\), the representation \(\pi_ i p_{ij} = \sum_{\widehat c \in {\mathbf C}} w_ cJ_ c (i,j)\), \(i,j \in S\), holds. Here \({\mathbf C}\) denotes the collection of all cycles in \(S\), \(w_ c\) is a nonnegative constant (actually the a.s. limit as \(n \to \infty\) of the relative frequency of \(\widehat c\) along a sample path until time \(n)\), and \(J_ c (i,j)\) is 0 or 1 according to whether or not \((i,j)\) is an edge of \(c\). This result due to Minping Qian and Min Qian [Z. Wahrscheinlichkeitstheorie Verw. Geb. 59, 203-210 (1982; Zbl 0483.60060)] together with a paper by J. MacQueen [Ann. Probab. 9, 604-610 (1981; Zbl 0464.60070)] on finite Markov chains defined by circuits are the starting point of an about 10 year long work of the author on various properties of Markov chains in terms of cycles, and the present book is mainly based on that work.
The book is divided into two parts. Part I entitled “Fundamentals of the cycle representations of Markov processes” contains seven chapters: 1. Directed circuits; 2. Genesis of Markov chains by circuits: the circuit chains; 3. Cycle representations of recurrent denumerable Markov chains; 4. Circuit representations of finite recurrent Markov chains; 5. Continuous parameter circuit processes with finite state space; 6. Spectral theory of circuit processes; 7. Higher-order circuit processes. Part II entitled “Applications of the cycle representations” contains three chapters: 1. Stochastic properties in terms of circuits; 2. Lévy’s theorem concerning positiveness of transition probabilities; 3. The rotational theory of Markov processes.
Most of the titles of the chapters are self-explanatory. Let us specify that the applications concern (1) the entropy production of a Markov chain and how it may be used to measure the distance from reversibility, and (2) the possibility to define an irreducible finite stochastic matrix by a rotation of the circle and a partition whose elements consist of finite unions of circle arcs.
In the reviewer’s opinion this is a very useful monograph which avoids ready ways and opens new research perspectives. It will certainly stimulate further work especially on the interplay of algebraic and geometrical aspects of Markovian dependence and its generalizations.

MSC:

60-02 Research exposition (monographs, survey articles) pertaining to probability theory
60J10 Markov chains (discrete-time Markov processes on discrete state spaces)
60B99 Probability theory on algebraic and topological structures
60G05 Foundations of stochastic processes
60J05 Discrete-time Markov processes on general state spaces
60J25 Continuous-time Markov processes on general state spaces
05C38 Paths and cycles
PDFBibTeX XMLCite