zbMATH — the first resource for mathematics

A generalized class of certainty and information measures. (English) Zbl 0553.94005
The objective of the paper is to introduce a generalized probabilistic theory of discrete information measures. Within this framework, information measures are developed with the help of certainty measures.
The authors give a definition of a generalized measure of average certainty. In the characterization theorem, the property of multiplicativity (for independent stochastic experiments) plays an important role. Other properties of the certainty measure are also studied. As a special case, the quadratic certainty measure (information energy) is obtained: \(\sum^{n}_{i=1}p^ 2_ i\), for \(p_ i\geq 0\), \(i=1,...,n,\) \(\sum^{n}_{i=1}p_ i=1.\)
Another section of the paper deals with the relation between certainty and information, and discusses the various methods in order to obtain measures of information based on the certainty measure. The characterization theorem for the information measures H(X) makes use of the following axiom: for stochastically independent experiments X, Y it holds that \(H(XY)=H(X)+H(Y)+cH(X)H(Y),\) \(c\in {\mathbb{R}}\). This axiom (pseudoadditivity) is a direct consequence of the multiplicativity property of the certainty measure. Properties of the general classes of information measures are also studied. As special cases, there may be obtained most of the presently known information measures.
Reviewer: R.Andonie

94A17 Measures of information, entropy
Full Text: DOI
[1] Aczel, J., On different characterizations of entropies, () · Zbl 0209.50801
[2] Arimoto, S., Information theoretical considerations on estimation problems, Inform. and control, 19, 181-194, (1971) · Zbl 0222.94022
[3] J. Bernoulli, Ars Conjectandi, Basileae, 1713.
[4] Bhargava, T.N.; Doyle, P.H., A geometric study of diversity, J. theoret. biol., 43, 241-251, (1974) · Zbl 0293.10006
[5] Boekee, D.E.; van der Lubbe, J.C.A., Some aspects of error bounds in feature selection, Pattern recognition, 11, 353-360, (1979) · Zbl 0433.62037
[6] Boekee, D.E.; van der Lubbe, J.C.A., The R-norm information measure, Inform. and control, 45, 136-155, (1980) · Zbl 0455.94005
[7] Bruckmann, G., Einige bemerkungen zur statistischen messung der konzentration, Metrika, 14, 183-213, (1969)
[8] Chen, C.H., On information and distance measures, error bounds and feature selection, Inform. sci., 10, 159-173, (1976) · Zbl 0333.94007
[9] Csiszar, I., Information measures: A critical survey, (), 73-86 · Zbl 0401.94010
[10] Daroczy, Z., Generalized information functions, Inform. and control, 16, 36-51, (1970) · Zbl 0205.46901
[11] Devijver, P.A., Entropie quadratique et reconnaissance des formes, (), 257-258 · Zbl 0378.94020
[12] Hardy, G.H.; Littlewood, J.E.; Polya, G., Inequalities, (1973), Cambridge U.P London · Zbl 0634.26008
[13] Havrda, J.; Charvat, F., Quantification method of classification processes, concept of structural a-entropy, Kybernetika, 3, 30-35, (1967) · Zbl 0178.22401
[14] Leibniz, G.W.F., (), Geneva
[15] Leti, G., Sull’entropia, su un indice del gini e su altre misure dell’eterogeneita di un collettivo, Metron, 24, 332-378, (1965)
[16] Nath, P., Entropy, inaccuracy and information, Metrika, 13, 136-148, (1968) · Zbl 0187.16902
[17] Onicescu, O., Energie informationelle, C.R. acad. sci. Paris ser. A, 263, 841-842, (28 Nov. 1966)
[18] Pielou, E.C.; Pielou, E.C., An introduction to mathematical ecology, (1969), Wiley New York, revised edition of · Zbl 0259.92001
[19] Rathie, P.N., Generalized entropies in coding theory, Metrika, 18, 216-219, (1972) · Zbl 0209.22004
[20] Renyi, A., On measures of entropy and information, (), 547-561, No. 1
[21] Shannon, C.E.; Shannon, C.E., The mathematical theory of communication, Bell. system tech. J., Bell. system tech. J., 27, 623-656, (1948) · Zbl 1154.94303
[22] Sharma, B.D., On amount of information of type-β and other measures, Metrika, 19, 1-10, (1972) · Zbl 0268.94007
[23] Sharma, B.D.; Mittal, D.P., New non-additive measures of entropy for discrete probability distributions, J. math. sci., 10, 28-40, (1975)
[24] Sheng, C.L.; Shiva, S.G.S., On measure of information, (), 798-803
[25] Shiva, S.G.S.; Ahmed, N.U.; Georganas, N.D., Order preserving measures of information, J. appl. probab., 10, 666-670, (1973) · Zbl 0279.94016
[26] Simpson, E.H., Measurement of diversity, Nature, 163, 688, (1949) · Zbl 0032.03902
[27] Vajda, I., Bounds on the minimal error probability on checking a finite or countable number of hypotheses, Problemy peredachi informatsii, 4, 9-19, (1968)
[28] van der Lubbe, J.C.A., R-norm information and a general class of measures for certainty and information (in Dutch), () · Zbl 0616.62150
[29] van der Lubbe, J.C.A., On measures of certainty, (), 136-137 · Zbl 0616.62150
[30] van der Lubbe, J.C.A.; Boekee, D.E., On measures of certainty and information in sequential and nonsequential hypothesis testing, (), 25-26
[31] van der Lubbe, J.C.A., A generalized probabilistic theory of the measurement of certainty and information, () · Zbl 0616.62150
[32] Watanabe, S., Knowing and guessing, a quantitative study of inference and information, (1969), Wiley New York · Zbl 0206.20901
[33] Weaver, W., Science and imagination, selected papers of warren weaver, (1967), Basic Books New York
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.