zbMATH — the first resource for mathematics

Asymptotics of the Shannon and Renyi entropies for sums of independent random variables. (English. Russian original) Zbl 0921.62005
Probl. Inf. Transm. 34, No. 3, 219-232 (1998); translation from Probl. Peredachi Inf. 34, No. 3, 17-31 (1998).
Let \(X_1,X_2,\dots\) be i.i.d. integer valued random variables with finite variance. Let
\(| E\exp(itX_1) |<1\), \(t\neq 0\). The authors study the Shannon and Renyi entropies \[ H_n=-\sum_n P_n(k)\log P_n(k)\quad\text{and}\quad(1-\alpha)^{-1}\log\sum_k(P_n (k))^\alpha,\;\alpha>1, \] of \(X_1+\cdots +X_n\) as \(n\to\infty\), where \(P_n(k)=P(X_1+ \cdots+X_n=k)\).
It is shown from the local limit theorem for \(P_n(k)\), that \(H_n=2^{-1}\log n+O(1)\) and \(H_n (\alpha)= 2^{-1}\log n+O(1)\). When \(E| X_1|^N <\infty\) with \(N\geq 3\), asymptotic expansions of the form \(2^{-1}\log n+\sum_k C_kn^{-k}\) with \(2k\leq N-2\) are proved by hard estimates, using the Edgeworth expansion of \(P_n(k)\). The \(c_k\) depend on the semi-invariants of \(X_1\), with \(c_0=2^{-1} \log(2\pi e \sigma^2)\) for Shannon and \(c_0=2^{-1}\log (2\pi\sigma^2) +2^{-1} (\alpha-1)^{-1} \log\alpha\) for Renyi, \(c_1\) and \(c_2\) are computed generally and some \(c_k\) with \(k\geq 3\) for special distributions. The corresponding theorems for absolutely continuous distributions are stated.
Reviewer: A.J.Stam (Winsum)

62B10 Statistical aspects of information-theoretic topics
60G50 Sums of independent random variables; random walks
62E20 Asymptotic distribution theory in statistics
60F99 Limit theorems in probability theory
94A17 Measures of information, entropy