zbMATH — the first resource for mathematics

Comparison of maximum entropy and higher-order entropy estimators. (English) Zbl 1043.62001
Summary: We show that generalized maximum entropy (GME) is the only estimation method that is consistent with a set of five axioms. The GME estimator can be nested using a single parameter, \(\alpha\), into two more general classes of estimators: GME-\(\alpha\) estimators. Each of these GME-\(\alpha\) estimators violates one of the five basic axioms. However, small-sample simulations demonstrate that certain members of these GME-\(\alpha\) classes of estimators may outperform the GME estimation rule.

62B10 Statistical aspects of information-theoretic topics
62F10 Point estimation
62P20 Applications of statistics to economics
Full Text: DOI
[1] Curado, E. M. F.; Tsallis, C.: Generalized statistical mechanics: connection with thermodynamics. Journal of physics A 24, L69-L72 (1991)
[2] Csiszar, I.: Why least squares and maximum entropy? an axiomatic approach to inference for linear inverse problems. The annals of statistics 19, 2032-2066 (1991) · Zbl 0753.62003
[3] Ferretti, N.; Kelmansky, D.; Yohai, V. J.; Zamar, R. H.: A class of locally and globally robust regression estimates. Journal of the American statistical association 94, 174-188 (1999) · Zbl 1072.62531
[4] Golan, A.: A simultaneous estimation and variable selection rule. Journal of econometrics 101, 165-193 (2001) · Zbl 0967.62098
[5] Golan, A., 2002. Information and entropy econometrics–editor’s view. Journal of Econometrics, this issue. · Zbl 1088.62519
[6] Golan, A.; Judge, G.; Miller, D.: Maximum entropy econometrics: robust estimation with limited data. (1996) · Zbl 0884.62126
[7] Holste, D.; Grobe, I.; Herzel, H.: Bayes estimators of generalized entropies. Journal of physics A 31, 2551-2566 (1998) · Zbl 0942.62003
[8] Imbens, G. W.; Johnson, P.; Spady, R. H.: Information-theoretic approaches to inference in moment condition models. Econometrica 66, 333-357 (1998) · Zbl 1055.62512
[9] Jaynes, E. T.: Information theory and statistical mechanics. Physics review 106, 620-630 (1957) · Zbl 0084.43701
[10] Jaynes, E. T.: Information theory and statistical mechanics II. Physics review 108, 171-190 (1957) · Zbl 0084.43701
[11] Kitamura, Y.; Stutzer, M.: An information-theoretic alternative to generalized method of moment estimation. Econometrica 66, No. 4, 861-874 (1997) · Zbl 0894.62011
[12] Levine, R. D.: An information theoretical approach to inversion problems. Journal of physics A 13, 91-108 (1980) · Zbl 0421.60098
[13] Owen, A.: Empirical likelihood for linear models. The annals of statistics 19, 1725-1747 (1991) · Zbl 0799.62048
[14] Pukelsheim, F.: The three-sigma rule. The American statistician 48, No. 4, 88-91 (1994)
[15] Qin, J.; Lawless, J.: Empirical likelihood and general estimating equations. The annals of statistics 22, 300-325 (1994) · Zbl 0799.62049
[16] Renyi, A., 1970. Probability Theory. North-Holland, Amsterdam. · Zbl 0245.60002
[17] Retzer, J., Soofi, E., 2002. Information indices: unification and applications. Journal of Econometrics, this issue. · Zbl 1051.62006
[18] Shannon, C. E.: A mathematical theory of communication. Bell system technical journal 27, 379-423 (1948) · Zbl 1154.94303
[19] Shore, J. E.; Johnson, R. W.: Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE transactions on information theory 26, No. 1, 26-37 (1980) · Zbl 0429.94011
[20] Skilling, J.: The axioms of maximum entropy. Maximum entropy and Bayesian methods in science and engineering, 173-187 (1988)
[21] Skilling, J.: Classic maximum entropy. Maximum entropy and Bayesian methods in science and engineering, 45-52 (1989) · Zbl 0850.62124
[22] Tsallis, C.: Possible generalization of Boltzmann–Gibbs statistics. Journal of statistical physics 52, 479-487 (1988) · Zbl 1082.82501
[23] Zellner, A., 1997. The Bayesian method of moments (BMOM): theory and applications. In: Fomby, T., Hill, R.C. (Eds.), Advances in Econometrics. JAI Press, London, UK.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.