The strong ergodic theorem for densities: Generalized Shannon-McMillan- Breiman theorem. (English) Zbl 0608.94001

Barron, Andrew R.

Ann. Probab. 13, 1292-1303 (1985).

Let \((X_ 1,X_ 2,...)\) be a stationary process with probability densities \(f(X_ 1,X_ 2,...,X_ n)\) with respect to Lebesgue measure or with respect to a Markov measure with a stationary transition measure. It is shown that the sequence of relative entropy densities (1/n)log f(X\({}_ 1,X_ 2,...,X_ n)\) converges almost surely. This long- conjectured result extends the \(L^ 1\) convergence obtained by Moy, Perez, and Kieffer and generalizes the Shannon-McMillan-Breiman theorem to nondiscrete processes. The heart of the proof is a new martingale inequality which shows that logarithms of densities are \(L^ 1\) dominated.

MSC:

94A17	Measures of information, entropy
28D05	Measure-preserving transformations
60G42	Martingales with discrete parameter
62B10	Statistical aspects of information-theoretic topics
60F15	Strong limit theorems
60G10	Stationary stochastic processes

Keywords:

stationary ergodic process; Moy-Perez theorem; asymptotic equipartition property; relative entropy densities; nondiscrete processes; martingale inequality; logarithms of densities