zbMATH — the first resource for mathematics

Solution of Shannon’s problem on the monotonicity of entropy. (English) Zbl 1062.94006
The authors have solved Shannon’s problem on the monotonocity of entropy. This has been conjectured in 1978, but a proof has never been given. They explore the underlying structure of sums of independent random variables and use it to prove that if \(X_i\), \(i = 1,2,\dots,\) are i.i.d. square-integrable random variables, the entropy of the normalized sum satisfies \[ \text{Ent} \left({X_1 + \dots + X_n \over \sqrt n}\right) \leq \text{Ent} \left({X_1 + \dots + X_{n+1} \over \sqrt{n + 1}}\right) \] where \(\text{Ent}(X) = - \int_k f \log f\). This result is generalized for non-identically distributed random variables as well as for the random vector.

94A17 Measures of information, entropy
62B10 Statistical aspects of information-theoretic topics
PDF BibTeX Cite
Full Text: DOI
[1] D. Bakry and Michel Émery, Diffusions hypercontractives, Séminaire de probabilités, XIX, 1983/84, Lecture Notes in Math., vol. 1123, Springer, Berlin, 1985, pp. 177 – 206 (French). · Zbl 0561.60080
[2] K. Ball, F. Barthe, and A. Naor. Entropy jumps in the presence of a spectral gap. Duke Math. J., 119(1):41-63, 2003. · Zbl 1036.94003
[3] Andrew R. Barron, Entropy and the central limit theorem, Ann. Probab. 14 (1986), no. 1, 336 – 342. · Zbl 0599.60024
[4] E. A. Carlen and A. Soffer, Entropy production by block variable summation and central limit theorems, Comm. Math. Phys. 140 (1991), no. 2, 339 – 371. · Zbl 0734.60024
[5] Elliott H. Lieb, Proof of an entropy conjecture of Wehrl, Comm. Math. Phys. 62 (1978), no. 1, 35 – 41. · Zbl 0385.60089
[6] C. E. Shannon and W. Weaver. The mathematical theory of communication. University of Illinois Press, Urbana, IL, 1949. · Zbl 0041.25804
[7] A. J. Stam, Some inequalities satisfied by the quantities of information of Fisher and Shannon, Information and Control 2 (1959), 101 – 112. · Zbl 0085.34701
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.