The strong ergodic theorem for densities: Generalized Shannon-McMillan- Breiman theorem. (English) Zbl 0608.94001

Let \((X_ 1,X_ 2,...)\) be a stationary process with probability densities \(f(X_ 1,X_ 2,...,X_ n)\) with respect to Lebesgue measure or with respect to a Markov measure with a stationary transition measure. It is shown that the sequence of relative entropy densities (1/n)log f(X\({}_ 1,X_ 2,...,X_ n)\) converges almost surely. This long- conjectured result extends the \(L^ 1\) convergence obtained by Moy, Perez, and Kieffer and generalizes the Shannon-McMillan-Breiman theorem to nondiscrete processes. The heart of the proof is a new martingale inequality which shows that logarithms of densities are \(L^ 1\) dominated.


94A17 Measures of information, entropy
28D05 Measure-preserving transformations
60G42 Martingales with discrete parameter
62B10 Statistical aspects of information-theoretic topics
60F15 Strong limit theorems
60G10 Stationary stochastic processes
Full Text: DOI