On the mean and the variance of estimates of Kullback information and relative ”useful” information measures. (English) Zbl 0581.94004

Let \(P=(p_ 1,p_ 2,...,p_ n)\), \(0<p_ i<1\), \(\sum^{n}_{i=1}p_ i=1\) be a finite probability distribution of a set of events \((E_ 1,E_ 2,...,E_ n)\) on the basis of an experiment whose predicted probability distribution is \(Q=(q_ 1,q_ 2,...,q_ n)\), \(0<q_ i<1\), \(\sum^{n}_{i=1}q_ i=1\). Then Kullback’s measure of relative information is defined as \[ (1)\quad I(P/Q)=\sum^{n}_{i=1}p_ i \log (p_ i/q_ i). \] Introducing a ”utility” distribution \(U=(u_ 1,u_ 2,...,u_ n)\), where each \(u_ i>0\) measures the utility of the i-th outcome \(E_ i\), H. C. Taneja and R. K. Tuteja [Inf. Sci. 33, 217-222 (1984; Zbl 0558.94003)], in addition to the results of M. Beliş and S. Guiaşu [IEEE Trans. Inf. Theory IT- 14, 593-594 (1968)], considered the measure of relative ”useful” information: \[ (2)\quad I(P/Q)=\sum^{n}_{i=1}u_ ip_ i \log (p_ i/q_ i). \] In this paper, the mean and variance of the Maximum Likelihood Estimator (MLE) of (1) and (2) are obtained. It is shown that the MLE of (1) and (2) is, in fact, a consistent overestimator of their true values.
Reviewer: R.Andonie


94A17 Measures of information, entropy


Zbl 0558.94003
Full Text: EuDML


[1] T. W. Anderson: An Introduction to Multivariate Statistical Analysis. John Wiley and Sons Inc., New York, 1958. · Zbl 0083.14601
[2] M. Belis S. Guiasu: A Quantitative-Qualitative Measure of Information in Cybernetic Systems. IEEE Trans. Inform. Theory, 14 (1968), 593-594.
[3] S. Kullback: Information Theory and Statistics. Wiley, New York, 1959. · Zbl 0088.10406
[4] G. Longo: A Quantiative-Qualitative Measure of Information. Springer-Verlag, 1972.
[5] H. C. Taneja R. K. Tuteja: Characterization of a Quantitative-Qualitative Measure of Relative Information. Information Sciences, 33 (1984). · Zbl 0558.94003
[6] H. Theil: Economics and Information Theory. North-Holland Publishing Co., Amsterdam, 1967.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.