A sandwich proof of the Shannon-McMillan-Breiman theorem. (English) Zbl 0653.28013

Algoet, Paul H.; Cover, Thomas M.

Ann. Probab. 16, No. 2, 899-909 (1988).

Let \(X_ 0,X_ 1,..\). be a stationary ergodic process with Polish state space S. Let M be a reference measure on the space of all one-sided infinite sequences from S. Suppose that M is finite order Markov with stationary transition kernel. Further, suppose that for each positive integer n, the distribution \(P_ n\) of \((X_ 0,X_ 1,...,X_{n-1})\) is absolutely continuous with respect to the marginal distribution \(M_ n\) that one obtains from M by projecting on the first n coordinates. Let \(\{f_ n\}\) be the sequence of random variables \(f_ n=(dP_ n/dM_ n)(X_ 0,x_ 1,...,X_{n-1}),\quad n=1,2,...\quad.\) It is known that the sequence \(\{n^{-1} \log f_ n\}\) converges almost surely. (This result is a generalization of the Shannon-McMillan-Breiman theorem.) The authors provide a new proof of this result which involves an ingenious “sandwiching” of the limit superior and limit inferior of \(\{n^{-1} \log f_ n\}\) between two random variables that have the same expected value.

Reviewer: J.C.Kieffer

MSC:

28D05	Measure-preserving transformations
94A17	Measures of information, entropy
60F15	Strong limit theorems

Keywords:

ergodic theorem of information theory; relative entropy rate; stationary ergodic process; Polish state space; reference measure; marginal distribution; generalization of the Shannon-McMillan-Breiman theorem; sandwiching