×

Characterizations of non-normalized discrete probability distributions and their application in statistics. (English) Zbl 1493.62056

Summary: From the distributional characterizations that lie at the heart of Stein’s method we derive explicit formulae for the mass functions of discrete probability laws that identify those distributions. These identities are applied to develop tools for the solution of statistical problems. Our characterizations, and hence the applications built on them, do not require any knowledge about normalization constants of the probability laws. To demonstrate that our statistical methods are sound, we provide comparative simulation studies for the testing of fit to the Poisson distribution and for parameter estimation of the negative binomial family when both parameters are unknown. We also consider the problem of parameter estimation for discrete exponential-polynomial models which generally are non-normalized.

MSC:

62E10 Characterization and structure theory of statistical distributions
62G10 Nonparametric hypothesis testing
PDFBibTeX XMLCite
Full Text: DOI arXiv Link

References:

[1] ARAGÓN, J., EBERLY, D. and EBERLY, S. (1992). Existence and uniqueness of the maximum likelihood estimator for the two-parameter negative binomial distribution. Statistics & Probability Letters 15 375-379. · Zbl 0758.62017
[2] ARRAS, B. and HOUDRÉ, C. (2019). On Stein’s method for infinitely divisible laws with finite first moment. SpringerBriefs in Probability and Mathematical Statistics. Springer International Publishing, Cham. · Zbl 1447.60052
[3] BARBOUR, A. D. (1988). Stein’s method and Poisson process convergence. Journal of Applied Probability 25 175-184. · Zbl 0661.60034
[4] BARBOUR, A. D. (1990). Stein’s method for diffusion approximations. Probability Theory and Related Fields 84 297-322. · Zbl 0665.60008
[5] BARINGHAUS, L. and HENZE, N. (1992). A goodness of fit test for the Poisson distribution based on the empirical generating function. Statistics & Probability Letters 13 269-274. · Zbl 0741.62043
[6] BARP, A., BRIOL, F. X., DUNCAN, A. B., GIROLAMI, M. A. and MACKEY, L. W. (2019). Minimum Stein discrepancy estimators. In Proceedings of the 33rd International Conference on Neural Information Processing Systems (H. WALLACH, H. LAROCHELLE, A. BEYGELZIMER, F. D’ALCHÉ-BUC, E. FOX and R. GARNETT, eds.). Advances in Neural Information Processing Systems 32 12964-12976. Curran Associates, Inc.
[7] BELTRÁN-BELTRÁN, J. I. and O’REILLY, F. J. (2019). On goodness of fit tests for the Poisson, negative binomial and binomial distributions. Statistical Papers 60 1-18. · Zbl 1455.62093
[8] BETSCH, S. and EBNER, B. (2021). Fixed point characterizations of continuous univariate probability distributions and their applications. Annals of the Institute of Statistical Mathematics 73 31-59. · Zbl 1472.62025
[9] BETSCH, S., EBNER, B. and KLAR, B. (2021). Minimum \[{L^q}\]-distance estimators for non-normalized parametric models. The Canadian Journal of Statistics 42 514-548.
[10] BROWN, T. C. and PHILLIPS, M. J. (1999). Negative binomial approximation with Stein’s method. Methodology and Computing in Applied Probability 1 407-421. · Zbl 0992.62015
[11] BYRD, R. H., LU, P., NOCEDAL, J. and ZHU, C. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing 16 1190-1208. · Zbl 0836.65080
[12] CHEN, L. H. Y. (1975). Poisson approximation for dependent trials. Annals of Probability 3 534-545. · Zbl 0335.60016
[13] CHEN, L. H. Y., GOLDSTEIN, L. and SHAO, Q. M. (2011). Normal approximation by Steins method. Probability and its applications. Springer, Berlin.
[14] CHOUDARY, A. D. R. and NICULESCU, C. P. (2014). Real Analysis on Intervals. Springer India, New Delhi.
[15] CHWIALKOWSKI, K., STRATHMANN, H. and GRETTON, A. (2016). A kernel test of goodness of fit. In Proceedings of the 33rd International Conference on Machine Learning (M. F. BALCAN and K. Q. WEINBERGER, eds.). JMLR: W&CP 48 2606-2615. Proceedings of Machine Learning Research.
[16] DUTANG, C., GOULET, V. and PIGEON, M. (2008). actuar: an R package for actuarial science. Journal of Statistical Software 25.
[17] EHM, W. (1991). Binomial approximation to the Poisson binomial distribution. Statistics & Probability Letters 11 7-16. · Zbl 0724.60021
[18] EICHELSBACHER, P. and REINERT, G. (2008). Stein’s method for discrete Gibbs measures. The Annals of Applied Probability 18 1588-1618. · Zbl 1146.60011
[19] FREY, J. (2012). An exact Kolmogorov-Smirnov test for the Poisson distribution with unknown mean. Journal of Statistical Computation and Simulation 82 1023-1033. · Zbl 1431.62187
[20] GORHAM, J. and MACKEY, L. (2015). Measuring sample quality with Stein’s method. In Proceedings of the 28th International Conference on Neural Information Processing Systems (C. CORTES, N. D. LAWRENCE, D. D. LEE, M. SUGIYAMA and R. GARNETT, eds.). Advances in Neural Information Processing Systems 28 226-234. Curran Associates, Inc.
[21] GÖTZE, F. (1991). On the rate of convergence in the multivariate CLT. The Annals of Probability 19 724-739. · Zbl 0729.62051
[22] GÜRTLER, N. and HENZE, N. (2000). Recent and classical goodness-of-fit tests for the Poisson distribution. Journal of Statistical Planning and Inference 90 207-225. · Zbl 0958.62043
[23] GUTMANN, M. U. and HYVÄRINEN, A. (2010). Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS) (Y. W. TEH and M. TITTERINGTON, eds.). JMLR: W&CP 9 297-304. Journal of Machine Learning Research - Proceedings Track.
[24] GUTMANN, M. U. and HYVÄRINEN, A. (2012). Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. Journal of Machine Learning Research 13 307-361. · Zbl 1283.62064
[25] HAYAKAWA, J. and TAKEMURA, A. (2016). Estimation of exponential-polynomial distribution by holonomic gradient descent. Communications in Statistics - Theory and Methods 45 6860-6882. · Zbl 1349.60013
[26] HENZE, N. (1996). Empirical-distribution-function goodness-of-fit tests for discrete models. The Canadian Journal of Statistics / La Revue Canadienne de Statistique 24 81-93. · Zbl 0846.62037
[27] HINTON, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation 14 1771-1800. · Zbl 1010.68111
[28] HYVÄRINEN, A. (2005). Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research 6 695-709. · Zbl 1222.62051
[29] HYVÄRINEN, A. (2007). Some extensions of score matching. Computational Statistics & Data Analysis 51 2499-2512. · Zbl 1161.62326
[30] INGLOT, T. (2019). Data driven efficient score tests for Poissonity. Probability and Mathematical Statistics 39 115-126. · Zbl 1422.62162
[31] JOHNSON, N. L., KOTZ, S. and KEMP, A. W. (1993). Univariate discrete distributions (2nd Edition). Wiley Series in Probability and Mathematical Statistics. Wiley, New York.
[32] KHMALADZE, E. (2013). Note on distribution free testing for discrete distributions. The Annals of Statistics 41 2979-2993. · Zbl 1294.62095
[33] KLAR, B. (1999). Goodness-of-fit tests for discrete models based on the integrated distribution function. Metrika 49 53-69. · Zbl 1093.62533
[34] KYRIAKOUSSIS, A., LI, G. and PAPADOPOULOS, A. (1998). On characterization and goodness-of-fit test of some discrete distribution families. Journal of Statistical Planning and Inference 74 215-228. · Zbl 0945.62018
[35] LEDWINA, T. and WYLUPEK, G. (2017). On Charlier polynomials in testing Poissonity. Communications in Statistics - Simulation and Computation 46 1918-1932. · Zbl 1364.62047
[36] LEY, C., REINERT, G. and SWAN, Y. (2017). Stein’s method for comparison of univariate distributions. Probability Surveys 14 1-52. · Zbl 1406.60010
[37] LEY, C. and SWAN, Y. (2011). A unified approach to Stein characterizations. arXiv e-prints 1105.4925v3.
[38] LEY, C. and SWAN, Y. (2013a). Stein’s density approach and information inequalities. Electronic Communications in Probability 18 1-14. · Zbl 1307.60009
[39] LEY, C. and SWAN, Y. (2013b). Local Pinsker inequalities via Stein’s discrete density approach. IEEE Transactions on Information Theory 59 5584-5591. · Zbl 1364.94243
[40] LIU, Q., LEE, J. D. and JORDAN, M. (2016). A kernelized Stein discrepancy for goodness-of-fit tests. In Proceedings of the 33rd International Conference on Machine Learning (M. F. BALCAN and K. Q. WEINBERGER, eds.). JMLR: W&CP 48 276-284. Proceedings of Machine Learning Research.
[41] LYU, S. (2009). Interpretation and generalization of score matching. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. UAI’09 359-366. AUAI Press.
[42] MATSUDA, T. and HYVÄRINEN, A. (2019). Estimation of non-normalized mixture models. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS) (K. CHAUDHURI and M. SUGIYAMA, eds.). PMLR 89 2555-2563. Proceedings of Machine Learning Research.
[43] NIKITIN, Y. Y. (2017). Tests based on characterizations, and their efficiencies: A survey. Acta et Commentationes Universitatis Tartuensis de Mathematica 21 3-24. · Zbl 1372.60022
[44] PEKÖZ, E. A. (1996). Stein’s method for geometric approximation. Journal of Applied Probability 33 707-713. · Zbl 0865.60014
[45] PUIG, P. and WEISS, C. H. (2020). Some goodness-of-fit tests for the Poisson distribution with applications in biodosimetry. Computational Statistics & Data Analysis 144 106878. · Zbl 1504.62022
[46] RIZZO, M. and SZÉKELY, G. (2019). energy: e-statistics: multivariate inference via the energy of data. R package, version 1.7-7.
[47] RUEDA, R., O’REILLY, F. and PÉREZ-ABREU, V. (1991). Goodness of fit for the Poisson distribution based on the probability generating function. Communications in Statistics - Theory and Methods 20 3093-3110. · Zbl 0800.62093
[48] RUEDA, R. and O’REILLY, F. (1999). Tests of fit for discrete distributions based on the probability generating function. Communications in Statistics - Simulation and Computation 28 259-274. · Zbl 1054.62546
[49] STEIN, C. (1972). A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Volume 2: Probability Theory 583-602. University of California Press, Berkeley. · Zbl 0278.60026
[50] STEIN, C. (1986). Approximate computation of expectations. Lecture Notes - Monograph Series, Vol. 7. Institute of Mathematical Statistics, Hayward. · Zbl 0721.60016
[51] STEIN, C., DIACONIS, P., HOLMES, S. and REINERT, G. (2004). Use of exchangeable pairs in the analysis of simulations. In Stein’s Method (P. DIACONIS and S. HOLMES, eds.). Lecture Notes - Monograph Series 46 1-25. Institute of Mathematical Statistics, Beachwood.
[52] SZÉKELY, G. and RIZZO, M. (2004). Mean distance test of Poisson distribution. Statistics & Probability Letters 67 241-247. · Zbl 1096.62046
[53] TAKENOUCHI, T. and KANAMORI, T. (2017). Statistical inference with unnormalized discrete models and localized homogeneous divergences. Journal of Machine Learning Research 18 1-26. · Zbl 1440.62088
[54] R CORE TEAM (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna.
[55] UEHARA, M., MATSUDA, T. and KIM, J. K. (2020). Imputation estimators for unnormalized models with missing data. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS) (S. CHIAPPA and R. CALANDRA, eds.). PMLR 108 831-841. Proceedings of Machine Learning Research.
[56] UEHARA, M., KANAMORI, T., TAKENOUCHI, T. and MATSUDA, T. (2020). A unified statistically efficient estimation framework for unnormalized models. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS) (S. CHIAPPA and R. CALANDRA, eds.). PMLR 108 809-819. Proceedings of Machine Learning Research.
[57] WOLODZKO, T. (2019). extraDistr: additional univariate and multivariate distributions. R package, version 1.8.11.
[58] YANG, J., LIU, Q., RAO, V. and NEVILLE, J. (2018). Goodness-of-fit testing for discrete distributions via Stein discrepancy. In Proceedings of the 35th International Conference on Machine Learning (J. DY and A. KRAUSE, eds.). PMLR 80 5561-5570. Proceedings of Machine Learning Research.
[59] YU, S., DRTON, M. and SHOJAIE, A. (2019). Generalized score matching for non-negative data. Journal of Machine Learning Research 20 1-70. · Zbl 1489.62082
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.