A new entropy upper bound. (English) Zbl 1255.26011

Summary: Entropy, conditional entropy and mutual information for discrete-valued random variables play important roles in information theory (see [R. B. Ash, Information theory. Intersci. Tracts in Pure and Appl. Math. 19. N.Y.-Lond.-Sydney: Intersci. Publishers, a Division of John Wiley and Sons (1965; Zbl 0141.34904)] and [T. M. Cover and J. A. Thomas, Elements of information theory. 2nd ed. Wiley-Interscience. Hoboken, NJ: John Wiley & Sons. (2006; Zbl 1140.94001)]). Our purpose within this work is to present a strong upper bound for the classical Shannon entropy, refining recent results from the literature. For this purpose we have considered the work of S. Simic [Appl. Math. Lett. 22, No. 8, 1262–1265 (2009; Zbl 1173.26308)], where new entropy bounds based on a new refinement of Jensen’s inequality are presented. Our work improves the basic result of Simic through a stronger refinement of Jensen’s inequality which is then applied to information theory.


26D15 Inequalities for sums, series and integrals
94A17 Measures of information, entropy
26D10 Inequalities involving derivatives and differential and integral operators
Full Text: DOI


[1] Jensen, J.L.W.V., Sur LES fonctions convexes et LES inégalités entre LES valeurs moyennes, Acta math., 30, 175-193, (1906) · JFM 37.0422.02
[2] Rassias, T.M., Survey on classical inequalities, (2000), Kluwer Academic Publishers Dordrecht · Zbl 0998.00010
[3] Diaz-Barrero, J.L.; Popescu, P.G., Certain inequalities for convex functions, J. inequal. pure appl. math., 7, 2, 1-5, (2006) · Zbl 1132.26011
[4] Simic, S., Jensen’s inequality and new entropy bounds, Appl. math. lett., 22, 1262-1265, (2009) · Zbl 1173.26308
[5] Dragomir, S.S.; Goh, C.J., A counterpart of jensen’s discrete inequality for differentiable convex mappings and applications in information theory, Math. comput. modelling, 24, 1-11, (1996) · Zbl 0862.94010
[6] Dragomir, S.S.; Goh, C.J., Some bounds on entropy measures in information theory, Appl. math. lett., 10, 3, 23-28, (1997) · Zbl 0906.94004
[7] Matic, M.; Pearce, C.E.M.; Pecaric, J., Refinements of some bounds in information theory, Anziam j., 42, 387-398, (2001) · Zbl 0993.94008
[8] Ash, R., Information theory, (1965), Interscience New York · Zbl 0141.34904
[9] Cover, T.M.; Thomas, J.A., Elements of information theory, (2006), John Wiley and Sons, Inc. · Zbl 1140.94001
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.