×

Towards a better understanding of the dual representation of phi divergences. (English) Zbl 1401.62031

Summary: The aim of this paper is to study different estimation procedures based on \(\phi \)-divergences. The dual representation of \(\phi \)-divergences based on the Fenchel-Legendre duality provides a way to estimate \(\phi\)-divergences by a simple plug-in of the empirical distribution without any smoothing technique. Resulting estimators are thoroughly studied theoretically and with simulations showing that the so called minimum \(\phi \)-divergence estimator is generally non robust and behaves similarly to the maximum likelihood estimator. We give some arguments supporting the non robustness property, and give insights on how to modify the classical approach. An alternative class of robust estimators based on the dual representation of \(\phi \)-divergences is introduced. We study consistency and robustness properties from an influence function point of view of the new estimator. In a second part, we invoke the Basu-Lindsay approach for approximating \(\phi \)-divergences and provide a comparison between these approaches. The so called dual \(\phi \)-divergence is also discussed and compared to our new estimator. A full simulation study of all these approaches is given in order to compare efficiency and robustness of all mentioned estimators against the so-called minimum density power divergence, showing encouraging results in favor of our new class of minimum dual \(\phi \)-divergences.

MSC:

62F10 Point estimation
62F35 Robustness and adaptive procedures (parametric inference)
62F12 Asymptotic properties of parametric estimators
62G07 Density estimation

Software:

R
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Al Mohamad D (2015) Towards a better understanding of the dual representation of phi divergences. arXiv:1506.02166 · Zbl 1401.62031
[2] Ali, SM; Silvey, SD, A general class of coefficients of divergence of one distribution from another, J R Stat Soc Ser B, 28, 131-142, (1966) · Zbl 0203.19902
[3] Barron, AR; Sheu, CH, Approximation of density functions by sequences of exponential families, Ann Stat, 19, 1347-1369, (1991) · Zbl 0739.62027 · doi:10.1214/aos/1176348252
[4] Basu, A; Lindsay, BG, Minimum disparity estimation for continuous models: efficiency, distributions and robustness, Ann Inst Stat Math, 46, 683-705, (1994) · Zbl 0821.62018 · doi:10.1007/BF00773476
[5] Basu, A; Sarkar, S, The trade-off between robustness and efficiency and the effect of model smoothing in minimum disparity inference, J Stat Comput Simul, 50, 173-185, (1994) · doi:10.1080/00949659408811609
[6] Basu, A; Harris, IR; Hjort, NL; Jones, MC, Robust and efficient estimation by minimising a density power divergence, Biometrika, 85, 549-559, (1998) · Zbl 0926.62021 · doi:10.1093/biomet/85.3.549
[7] Beran, R, Minimum Hellinger distance estimates for parametric models, Ann Stat, 5, 445-463, (1977) · Zbl 0381.62028 · doi:10.1214/aos/1176343842
[8] Bordes, L; Vandekerkhove, P, Semiparametric two-component mixture model with a known component: an asymptotically normal estimator, Math Methods Stat, 19, 22-41, (2010) · Zbl 1282.62068 · doi:10.3103/S1066530710010023
[9] Bouezmarni, T; Rombouts, JV, Nonparametric density estimation for multivariate bounded data, J Stat Plan Inference, 140, 139-152, (2010) · Zbl 1178.62026 · doi:10.1016/j.jspi.2009.07.013
[10] Bouezmarni, T; Scaillet, O, Consistency of asymmetric kernel density estimators and smoothed histograms with application to income data, Econom Theory, 21, 390-412, (2005) · Zbl 1062.62058 · doi:10.1017/S0266466605050218
[11] Broniatowski, M, Minimum divergence estimators, maximum likelihood and exponential families, Stat Probab Lett, 93, 27-33, (2014) · Zbl 1400.62045 · doi:10.1016/j.spl.2014.06.014
[12] Broniatowski, M; Keziou, A, Minimization of divergences on sets of signed measures, Stud Sci Math Hungar, 43, 403-442, (2006) · Zbl 1121.28004
[13] Broniatowski, M; Keziou, A, Parametric estimation and tests through divergences and the duality technique, J Multivar Anal, 100, 16-36, (2009) · Zbl 1151.62023 · doi:10.1016/j.jmva.2008.03.011
[14] Broniatowski, M; Vajda, I, Several applications of divergence criteria in continuous families, Kybernetika, 48, 600-636, (2012) · Zbl 1318.62013
[15] Cherfi M (2011) Dual \(φ \)-divergences estimation in normal models. arXiv:1108.2999 · Zbl 1215.62038
[16] Csiszár, I, Eine informationstheoretische ungleichung und ihre anwendung auf den beweis der ergodizität von markoffschen ketten, Publ Math Inst Hungar Acad Sci, 8, 95-108, (1963) · Zbl 0124.08703
[17] Dempster, AP; Laird, NM; Rubin, DB, Maximum likelihood from incomplete data via the EM algorithm, J R Stat Soc Ser B, 39, 1-38, (1977) · Zbl 0364.62022
[18] Dobele-kpoka L, Baudin FG (2013) Non parametric method of mixed associated kernels and applications. Theses, Université de Franche-Comté. https://tel.archives-ouvertes.fr/tel-01124288
[19] Feller W (1971) An introduction to probability theory and its applications. Wiley mathematical statistics series, vol 2. Wiley, New York · Zbl 0219.60003
[20] Frýdlovà, I; Vajda, I; Kus, V, Modified power divergence estimators in normal models simulation and comparative study, Kybernetika, 48, 795-808, (2012) · Zbl 1318.62012
[21] Funke, B; Kawka, R, Nonparametric density estimation for multivariate bounded data using two non-negative multiplicative bias correction methods, Comput Stat Data Anal, 92, 148-162, (2015) · Zbl 1468.62057 · doi:10.1016/j.csda.2015.07.006
[22] Ghosh A, Basu A (2015) The minimum s-divergence estimator under continuous models: the basu-lindsay approach. Stat Pap 1-32
[23] Ghosh A, Harris IR, Maji A, Basu A, Pardo L (2013) A generalized divergence for statistical inference. Technical Report, Byesian and Interdisciplinary Research Unit Indian Statistical Institute · Zbl 1387.62041
[24] Jiménz, R; Shao, Y, On robustness and efficiency of minimum divergence estimators, Test, 10, 241-248, (2001) · Zbl 1014.62019 · doi:10.1007/BF02595695
[25] Karunamuni, R; Alberts, T, On boundary correction in kernel density estimation, Stat Methodol, 2, 191-212, (2005) · Zbl 1248.62051 · doi:10.1016/j.stamet.2005.04.001
[26] Kuchibhotla, AK; Basu, A, A general set up for minimum disparity estimation, Stat Probab Lett, 96, 68-74, (2015) · Zbl 1314.62089 · doi:10.1016/j.spl.2014.08.020
[27] Lavancier F, Rochet P (2016) A general procedure to combineestimators. Comput Stat Data Anal 94:175-192 · Zbl 1468.62110
[28] Liese, F; Vajda, I, On divergences and informations in statistics and information theory, IEEE Trans Inf Theory, 52, 4394-4412, (2006) · Zbl 1287.94025 · doi:10.1109/TIT.2006.881731
[29] Lindsay, BG, Efficiency versus robustness: the case for minimum Hellinger distance and related methods, Ann Stat, 22, 1081-1114, (1994) · Zbl 0807.62030 · doi:10.1214/aos/1176325512
[30] Meister A (2009) Deconvolution problems in nonparametric statistics. Lecture notes in statistics, Springer, Berlin · Zbl 1178.62028 · doi:10.1007/978-3-540-87557-4
[31] Mnatsakanov, R; Sarkisian, K, Varying kernel density estimation on \(\mathbb{R}_+\), Stat Probab Lett, 82, 1337-1345, (2012) · Zbl 1489.62114 · doi:10.1016/j.spl.2012.03.033
[32] Park, C; Basu, A, Minimum disparity estimation : asymptotic normality and breakdown point results, Bull Inform Cybern, 36, 19-33, (2004) · Zbl 1271.62075
[33] R Core Team (2015) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/
[34] Schuster, EF, Estimation of a probability density function and its derivatives, Ann Math Stat, 40, 1187-1195, (1969) · Zbl 0212.21703 · doi:10.1214/aoms/1177697495
[35] Silverman, BW, Weak and strong uniform consistency of the kernel estimate of a density and its derivatives, Ann Stat, 6, 177-184, (1978) · Zbl 0376.62024 · doi:10.1214/aos/1176344076
[36] Simpson, DG, Minimum Hellinger distance estimation for the analysis of count data, J Am Stat Assoc, 82, 802-807, (1987) · Zbl 0633.62029 · doi:10.1080/01621459.1987.10478501
[37] Tamura, RN; Boos, DD, Minimum Hellinger distance estimation for multivariate location and covariance, J Am Stat Assoc, 81, 223-229, (1986) · Zbl 0601.62051 · doi:10.1080/01621459.1986.10478264
[38] Toma, A; Broniatowski, M, Dual divergence estimators and tests: robustness results, J Multivar Anal, 102, 20-36, (2011) · Zbl 1206.62034 · doi:10.1016/j.jmva.2010.07.010
[39] Toma, A; Leoni-Aubin, S, Robust tests based on dual divergence estimators and saddlepoint approximations, J Multivar Anal, 101, 1143-1155, (2010) · Zbl 1185.62042 · doi:10.1016/j.jmva.2009.11.001
[40] Toma A, Leoni-Aubin S (2013) Optimal robust m-estimators using Rényi pseudodistances. J Multivar Anal 115(C):359-373 · Zbl 1297.62050
[41] van der Vaart A (1998) Asymptotic statistics, vol 3. Cambridge series in statistical and probabilistic mathematicscambridge University Press, Cambridge · Zbl 0910.62001 · doi:10.1017/CBO9780511802256
[42] Venables W, Ripley B (2013) Modern applied statistics with S. statistics and computing. Springer, New York · Zbl 1006.62003
[43] Wied, D; Weibßach, R, Consistency of the kernel density estimator: a survey, Stat Pap, 53, 1-21, (2012) · Zbl 1241.62049 · doi:10.1007/s00362-010-0338-1
[44] Zambom, AZ; Dias, R, A review of kernel density estimation with applications to econometrics, Int Econom Rev (IER), 5, 20-42, (2013)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.