# zbMATH — the first resource for mathematics

Asymptotics of the Shannon and Renyi entropies for sums of independent random variables. (English. Russian original) Zbl 0921.62005
Probl. Inf. Transm. 34, No. 3, 219-232 (1998); translation from Probl. Peredachi Inf. 34, No. 3, 17-31 (1998).
Let $$X_1,X_2,\dots$$ be i.i.d. integer valued random variables with finite variance. Let
$$| E\exp(itX_1) |<1$$, $$t\neq 0$$. The authors study the Shannon and Renyi entropies $H_n=-\sum_n P_n(k)\log P_n(k)\quad\text{and}\quad(1-\alpha)^{-1}\log\sum_k(P_n (k))^\alpha,\;\alpha>1,$ of $$X_1+\cdots +X_n$$ as $$n\to\infty$$, where $$P_n(k)=P(X_1+ \cdots+X_n=k)$$.
It is shown from the local limit theorem for $$P_n(k)$$, that $$H_n=2^{-1}\log n+O(1)$$ and $$H_n (\alpha)= 2^{-1}\log n+O(1)$$. When $$E| X_1|^N <\infty$$ with $$N\geq 3$$, asymptotic expansions of the form $$2^{-1}\log n+\sum_k C_kn^{-k}$$ with $$2k\leq N-2$$ are proved by hard estimates, using the Edgeworth expansion of $$P_n(k)$$. The $$c_k$$ depend on the semi-invariants of $$X_1$$, with $$c_0=2^{-1} \log(2\pi e \sigma^2)$$ for Shannon and $$c_0=2^{-1}\log (2\pi\sigma^2) +2^{-1} (\alpha-1)^{-1} \log\alpha$$ for Renyi, $$c_1$$ and $$c_2$$ are computed generally and some $$c_k$$ with $$k\geq 3$$ for special distributions. The corresponding theorems for absolutely continuous distributions are stated.
Reviewer: A.J.Stam (Winsum)

##### MSC:
 62B10 Statistical aspects of information-theoretic topics 60G50 Sums of independent random variables; random walks 62E20 Asymptotic distribution theory in statistics 60F99 Limit theorems in probability theory 94A17 Measures of information, entropy
##### Keywords:
entropy; local limit theorem