×
Wu, Yuefeng; Ghosal, Subhashis

Kullback Leibler property of kernel mixture priors in Bayesian density estimation. (English) Zbl 1135.62022

Electron. J. Stat. 2, 298-331 (2008); correction ibid. 3, 316-317 (2009).
Summary: Positivity of the prior probability of the Kullback-Leibler neighborhood around the true density, commonly known as the Kullback-Leibler property, plays a fundamental role in posterior consistency. A popular prior for Bayesian estimation is given by a Dirichlet mixture, where the kernels are chosen depending on the sample space and the class of densities to be estimated. The Kullback-Leibler property of the Dirichlet mixture prior has been shown for some special kernels like the normal density or Bernstein polynomials, under appropriate conditions.
We obtain easily verifiable sufficient conditions, under which a prior obtained by mixing a general kernel possesses the Kullback-Leibler property. We study a wide variety of kernels used in practice, including the normal, \(t\), histogram, Weibull, gamma densities and so on, and show that the Kullback-Leibler property holds if some easily verifiable conditions are satisfied at the true density. This gives a catalog of conditions required for the Kullback-Leibler property, which can be readily used in applications.

Cited in 1 Review
Cited in 32 Documents

MSC:

62F15 Bayesian inference
62G07 Density estimation

Keywords:

Dirichlet process; posterior consistency
Cite Review PDF
Full Text: DOI arXiv

References:

[1] Antoniak, C. (1974). Mixtures of Dirichlet processes with application to Bayesian non-parametric problems., Ann. Statist. 2 1152-1174. · Zbl 0335.60034 · doi:10.1214/aos/1176342871
[2] Arfken, G. (1985), Digamma and Polygamma Functions in Mathematical Methods for Physicists . 3rd ed. Academic Press, Orlando.
[3] Bouezmarni, T. and Scaillet, O. (2003). Consistency of asymmetric kernel density estimators and smoothed histograms with application to income data. DP0306, Institut de Statistique. Université Catholique de, Louvain. · Zbl 1062.62058
[4] Chen, S. (2000). Probability density function estimation using gamma kernels., Ann. Inst. Statist. Math. 52 No. 3, 471-480. · Zbl 0960.62038 · doi:10.1023/A:1004165218295
[5] Diaconis, P. and Ylvisaker, D. (1985). Quantifying prior opinion. In, Bayesian Statistics 2 (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, Eds.) 133-156. North-Holland, Amsterdam. · Zbl 0673.62004
[6] Escobar, M. and West, M. (1995). Bayesian density estimation and inference using mixtures., J. Amer. Statist. Assoc. 90 577-588. · Zbl 0826.62021 · doi:10.2307/2291069
[7] Escobar, M. and West, M. (1998). Computing nonparametric hierarchical models. In, Practical Nonparametric and Semiparametric Bayesian Statistics . Lecture Notes in Statistics , 133 1-22. Springer, New York. · Zbl 0918.62028 · doi:10.1007/978-1-4612-1732-9_1
[8] Feller, W. (1957)., An Introduction to Probability Theory and Its Application , Vol I. & II. John Wiley & Sons, Inc. · Zbl 0077.12201
[9] Ferguson, T. S. (1983). Bayesian density estimation by mixtures of normal distributions. In, Recent Advances in Statistics (M. Rizvi, J. Rustagi, and D. Siegmund, Eds.) 287-302. Academic Press, New York. · Zbl 0557.62030 · doi:10.1016/B978-0-12-589320-6.50018-6
[10] Ghosal, S. (2001). Convergence rates for density estimation with Bernstein polynomials., Ann. Statist. 29 1264-1280. · Zbl 1043.62024 · doi:10.1214/aos/1013203453
[11] Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999). Posterior consistency of Dirichlet mixtures in density estimation., Ann. Statist. 27 143-158. · Zbl 0932.62043 · doi:10.1214/aos/1018031105
[12] Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999). Consistent semiparametric Bayesian inference about a location parameter., J. Statist. Plann. Inference 77 181-193. · Zbl 1054.62528 · doi:10.1016/S0378-3758(98)00192-X
[13] Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities., Ann. Statist. 29 1233-1263. · Zbl 1043.62025 · doi:10.1214/aos/1013203452
[14] Ghosal, S. and van der Vaart, A. W. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities., Ann. Statist. 35 697-723. · Zbl 1117.62046 · doi:10.1214/009053606000001271
[15] Ghosal, S. and van der Vaart, A. W. (2009), Theory of Nonparametric Bayesian inference . Cambridge University Press (to appear).
[16] Ghosh, J. K. and Ramamoortrhi, R. V. (2003)., Bayesian Nonparametrics . Springer-Verlag, New York. · Zbl 1029.62004
[17] Ghosh, S. K. and Ghosal, S. (2006). Semiparametric accelerated failure time models for censored data. In, Bayesian Statistics and its Applications (S. K. Upadhyay et al., eds.) 213-229 Anamaya Publishers, New Delhi.
[18] Hason, T. (2006). Modeling censored lifetime data using a mixture of gammas baseline., Bayesian Analysis , 1 575-594. · Zbl 1331.62389 · doi:10.1214/06-BA119
[19] Kruijer, W. and van der Vaart. (2005). Posterior convergence rates for Dirichlet mixtures of beta densities., · Zbl 1134.62023
[20] Lavine, M. (1992). Some aspects of Polya tree distributions for statistical modeling., Ann. Statist. 20 1222-1235. · Zbl 0765.62005 · doi:10.1214/aos/1176348767
[21] Lo, A. Y. (1984). On a class of Bayessian nonparametric estimates I: density estimates., Ann. Statist. 1 38-53. · Zbl 0557.62036 · doi:10.1214/aos/1176346412
[22] Lorentz, G. (1953)., Bernstein Polynomials . University of Toronto Press, Toronto. · Zbl 0051.05001
[23] Petrone, S. (1999). Random Bernstein polynomials., Scand. J. Statist. 26 373-393. · Zbl 0939.62046 · doi:10.1111/1467-9469.00155
[24] Petrone, S. (1999). Bayesian density estimation using Bernstein polynomials., Canad. J. Statist. 27 105-126. · Zbl 0929.62044 · doi:10.2307/3315494
[25] Petrone, S. and Veronese, P. (2007). Feller operators and mixture priors in Bayesian nonparametrics., · Zbl 1191.62048
[26] Petrone, S. and Wasserman, L. (2002). Consistency of Bernstein polynomial posteriors., J. Roy. Statist. Soc., Ser. B 64 79-100. · Zbl 1015.62033 · doi:10.1111/1467-9868.00326
[27] Royden, H. L. (1988)., Real Analysis . Macmillan, New York; Collier MacMillan, London. · Zbl 0704.26006
[28] Schwartz, L. (1965). On Bayes procedures., Z. Wahrsch. Verw. Gebiete 4 10-26. · Zbl 0158.17606 · doi:10.1007/BF00535479
[29] Tokdar, S. (2006). Posterior consistency of Dirichlet location-scale mixture of normals in density estimation and regression., Sankhya: The Indian Journal of Statistics. 67 90-110. · Zbl 1193.62056
[30] West, M. (1992). Modeling with mixtures. In, Bayesian Statistcs 4 (J. M. Bernardo, J. O. Berger, A. P. David, and A. F. M. Smith, Eds.) 503-524. Oxford Univ. Press.
[31] West, M., Müller, P. and Escobar, M. (1994). Hierarchical priors and mixture models, with applications in regressions and density estimation. In, Aspects of Uncertainty: A Tribute to D. V. Lindley (P. R. Freeman and A. F. M. Smith, Eds.) 363-386. Wiley, Chichester. · Zbl 0842.62001
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.