×

Bayes estimate and inference for entropy and information index of fit. (English) Zbl 1482.62026

Summary: This article defines a quantized entropy and develops Bayes estimates and inference for the entropy and a Kullback-Leibler information index of the model fit. We use a Dirichlet process prior for the unknown data-generating distribution with a maximum entropy candidate model as the expected distribution. This formulation produces prior and posterior distributions for the quantized entropy, the information index of fit, the moments, and the model parameters. The posterior mean of the quantized entropy provides a Bayes estimate of entropy under quadratic loss. The consistency of the Bayes estimates and the information index are shown. The implementation and the performances of the Bayes estimates are illustrated using data simulated from exponential, gamma, and lognormal distributions.

MSC:

62B10 Statistical aspects of information-theoretic topics
62G99 Nonparametric inference
62P20 Applications of statistics to economics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Abramowitz M., Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables (1970) · Zbl 0171.38503
[2] Akaike H., IEEE Transactions on Automatic Control 19 pp 716– (1974) · Zbl 0314.62039 · doi:10.1109/TAC.1974.1100705
[3] Arizono I., The American Statistician 34 pp 20– (1989)
[4] Beirlant J., International Journal of Mathematical and Statistical Sciences 6 pp 17– (1997)
[5] Bernardo J. M., The Annals of Statistics 7 pp 686– (1979) · Zbl 0407.62002 · doi:10.1214/aos/1176344689
[6] Bowyer A., The Computer Journal 24 pp 162– (1981) · doi:10.1093/comjnl/24.2.162
[7] Campbell L. L., IEEE Transactions on Information Theory IT 41 pp 338– (1995) · Zbl 0822.94004 · doi:10.1109/18.370086
[8] Carota C., Journal of the American Statistical Association 91 pp 753– (1996) · doi:10.1080/01621459.1996.10476943
[9] Choi B., Statistics 40 pp 517– (2006) · Zbl 1120.62004 · doi:10.1080/02331880600822473
[10] Clarke B., Journal of the American Statistical Association 91 pp 173– (1996) · doi:10.1080/01621459.1996.10476674
[11] Clarke B., Journal of Statistical Planning and Inference 71 pp 137– (1998) · Zbl 0938.62028 · doi:10.1016/S0378-3758(98)00014-7
[12] Cover T. M., Elements of Information Theory (1991) · Zbl 0762.94001 · doi:10.1002/0471200611
[13] Dadpay A., Journal of Econometrics 138 pp 568– (2007) · Zbl 1418.62081 · doi:10.1016/j.jeconom.2006.05.010
[14] Dempster , A. P. ( 1974 ). The direct use of likelihood for significance testing . In: Proceedings of Conference on Foundational Questions in Statistical Inference . pp. 335 – 352 . · Zbl 0367.62004
[15] DeWaal D. J., South African Statistical Journal 30 pp 139– (1996)
[16] Dudewicz E. J., Journal of the American Statistical Association 76 pp 967– (1981) · doi:10.1080/01621459.1981.10477750
[17] Dudewicz E. J., New Perspectives in Theoretical and Applied Statistics pp 207– (1987)
[18] Ebrahimi N., Journal of Statistical Planning and Inference 64 pp 9– (1997) · Zbl 0904.62056 · doi:10.1016/S0378-3758(96)00215-7
[19] Ebrahimi N., IEEE Transactions on Reliability 47 pp 197– (1998) · Zbl 04548465 · doi:10.1109/24.722289
[20] Ebrahimi N., Annals of Institute of Statistical Mathematics 53 pp 325– (2001) · Zbl 1027.62025 · doi:10.1023/A:1012085320762
[21] Ebrahimi N., Journal of Royal Statistical Society B 54 pp 739– (1992)
[22] Ebrahimi N., Statistics and Probability Letters 20 pp 225– (1994) · Zbl 0805.62009 · doi:10.1016/0167-7152(94)90046-9
[23] Ebrahimi N., Journal of Econometrics 90 pp 317– (1999) · Zbl 1041.62501 · doi:10.1016/S0304-4076(98)00046-3
[24] Ferguson T. S., Annals of Statistics 2 pp 209– (1973) · Zbl 0255.62037 · doi:10.1214/aos/1176342360
[25] Gill C. A., Biometrika 66 (1979)
[26] Gokhale D. V., Computational Statistics and Data Analysis 1 pp 157– (1983) · Zbl 0567.62036 · doi:10.1016/0167-9473(83)90087-7
[27] Hall P., Annals of Institute of Mathematical Statistics 45 pp 69– (1993) · Zbl 0776.62038 · doi:10.1007/BF00773669
[28] Inverardi P. L. N., Communications in Statistics – Simulation and Computation 32 pp 17– (2003) · Zbl 1100.62519 · doi:10.1081/SAC-120013108
[29] Joe H., Annals of Institute of Mathematical Statistics 41 pp 683– (1989) · Zbl 0698.62042 · doi:10.1007/BF00057735
[30] Kraskov A., Physical Review E 69 (2004) · doi:10.1103/PhysRevE.69.066138
[31] Mazzuchi T. A., Computational Statistics and Data Analysis 32 pp 361– (2000) · Zbl 1023.62104 · doi:10.1016/S0167-9473(99)00090-0
[32] Mudholkar G. S., Journal of Statistical Planning and Inference 102 pp 211– (2002) · doi:10.1016/S0378-3758(01)00099-4
[33] Park S. G., Statistics and Probability Letters 44 pp 359– (1999) · Zbl 0968.62048 · doi:10.1016/S0167-7152(99)00027-9
[34] Park S. G., IEEE Transactions on Reliability 54 pp 22– (2005) · doi:10.1109/TR.2004.837314
[35] Park S., Journal of Statistical Computation and Simulation 73 pp 685– (2003) · Zbl 1033.62005 · doi:10.1080/0094965031000070367
[36] Parzen E., Statistical Science 19 pp 652– (2004) · Zbl 1100.62500 · doi:10.1214/088342304000000387
[37] Soofi E. S., Advances in Econometrics: Applying Maximum Entropy to Econometric Problems 12 pp 25– (1997)
[38] Soofi E. S., Journal of Econometrics 107 pp 17– (2002) · Zbl 1051.62006 · doi:10.1016/S0304-4076(01)00111-7
[39] Soofi E. S., Journal of the American Statistical Association 90 pp 657– (1995) · doi:10.1080/01621459.1995.10476560
[40] Spigelhalter D. J., Journal of the Royal Statistical Society, Series B 64 pp 583– (2002) · Zbl 1067.62010 · doi:10.1111/1467-9868.00353
[41] Taufer E., Communications in Statistics-Simulation and Computation 31 pp 189– (2002) · Zbl 1081.62501 · doi:10.1081/SAC-120003334
[42] Theil H., Economics Letters 5 pp 145– (1980) · doi:10.1016/0165-1765(80)90089-0
[43] Vasicek O., Journal of Royal Statistical Society B 38 pp 54– (1976)
[44] Yuan A., IEEE Transactions on Information Theory IT 45 pp 562– (1999) · Zbl 0946.94019 · doi:10.1109/18.749003
[45] Zellner A., Maximum Entropy and Bayesian Methods pp 17– (1991) · doi:10.1007/978-94-011-3460-6_2
[46] Zellner A., Journal of Econometrics 75 pp 51– (1996) · Zbl 0850.62910 · doi:10.1016/0304-4076(95)01768-2
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.