On the mean and the variance of estimates of Kullback information and relative ”useful” information measures. (English) Zbl 0581.94004

Taneja, Harish C.

Apl. Mat. 30, 166-175 (1985).

Let \(P=(p_ 1,p_ 2,...,p_ n)\), \(0<p_ i<1\), \(\sum^{n}_{i=1}p_ i=1\) be a finite probability distribution of a set of events \((E_ 1,E_ 2,...,E_ n)\) on the basis of an experiment whose predicted probability distribution is \(Q=(q_ 1,q_ 2,...,q_ n)\), \(0<q_ i<1\), \(\sum^{n}_{i=1}q_ i=1\). Then Kullback’s measure of relative information is defined as \[ (1)\quad I(P/Q)=\sum^{n}_{i=1}p_ i \log (p_ i/q_ i). \] Introducing a ”utility” distribution \(U=(u_ 1,u_ 2,...,u_ n)\), where each \(u_ i>0\) measures the utility of the i-th outcome \(E_ i\), H. C. Taneja and R. K. Tuteja [Inf. Sci. 33, 217-222 (1984; Zbl 0558.94003)], in addition to the results of M. Beliş and S. Guiaşu [IEEE Trans. Inf. Theory IT- 14, 593-594 (1968)], considered the measure of relative ”useful” information: \[ (2)\quad I(P/Q)=\sum^{n}_{i=1}u_ ip_ i \log (p_ i/q_ i). \] In this paper, the mean and variance of the Maximum Likelihood Estimator (MLE) of (1) and (2) are obtained. It is shown that the MLE of (1) and (2) is, in fact, a consistent overestimator of their true values.

Reviewer: R.Andonie

MSC:

94A17

Measures of information, entropy

Keywords:

relative information; utility; Maximum Likelihood Estimator

Citations:

Zbl 0558.94003

References:

[1]	T. W. Anderson: An Introduction to Multivariate Statistical Analysis. John Wiley and Sons Inc., New York, 1958. · Zbl 0083.14601
[2]	M. Belis S. Guiasu: A Quantitative-Qualitative Measure of Information in Cybernetic Systems. IEEE Trans. Inform. Theory, 14 (1968), 593-594.
[3]	S. Kullback: Information Theory and Statistics. Wiley, New York, 1959. · Zbl 0088.10406
[4]	G. Longo: A Quantiative-Qualitative Measure of Information. Springer-Verlag, 1972.
[5]	H. C. Taneja R. K. Tuteja: Characterization of a Quantitative-Qualitative Measure of Relative Information. Information Sciences, 33 (1984). · Zbl 0558.94003
[6]	H. Theil: Economics and Information Theory. North-Holland Publishing Co., Amsterdam, 1967.