×
Ishwaran, Hemant

Exponential posterior consistency via generalized Pólya urn schemes in finite semiparametric mixtures. (English) Zbl 0927.62047

Ann. Stat. 26, No. 6, 2157-2178 (1998).
Summary: Advances in Markov chain Monte Carlo (MCMC) methods now make it computationally feasible and relatively straightforward to apply the Dirichlet process prior in a wide range of Bayesian nonparametric problems. The feasibility of these methods rests heavily on the fact that the MCMC approach avoids direct sampling of the Dirichlet process and is instead based on sampling the finite-dimensional posterior which is obtained from marginalizing out the process.
In application, it is the integrated posterior that is used in the Bayesian nonparametric inference, so one might wonder about its theoretical properties. This paper presents some results in this direction. In particular, we will focus on a study of the posterior’s asymptotic behavior, specifically for the problem when the data is obtained from a finite semiparametric mixture distribution. A complication in the analysis arises because the dimension for the posterior, although finite, increases with the sample size. The analysis will reveal general conditions that ensure exponential posterior consistency for a finite dimensional parameter and which can be slightly generalized to allow the unobserved nonparametric parameters to be sampled from a generalized Pólya urn scheme. Several interesting examples are considered.

MSC:

62G20 Asymptotic properties of nonparametric inference
62F15 Bayesian inference
65C40 Numerical analysis or methods applied to Markov chains
62G05 Nonparametric estimation

Keywords:

Dirichlet process prior; exchangeability; Pólya urn scheme; posterior consistency; relative entropy; semiparametric mixture
Cite Review PDF
Full Text: DOI

References:

[1] ANTONIAK, C. E. 1974. Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Statist. 2 1152 1174. Z. · Zbl 0335.60034 · doi:10.1214/aos/1176342871
[2] BARRON, A. R. 1988. The exponential convergence of posterior probabilities with implications for Bay es estimators of density functions. Technical Report 7, Dept. Statistics, Univ. Illinois, Champaign. Z.
[3] BARRON, A. R. 1989. Uniformly powerful goodness of fit tests. Ann. Statist. 17 107 124. Z. · Zbl 0674.62032 · doi:10.1214/aos/1176347005
[4] BARRON, A. R., SCHERVISH, M. and WASSERMAN, L. 1998. The consistency of posterior distributions in nonparametric problems. Unpublished manuscript. Z. · Zbl 0980.62039 · doi:10.1214/aos/1018031206
[5] BLACKWELL, D. and MACQUEEN, J. B. 1973. Ferguson distributions via Poly a urn schemes. Ánn. Statist. 1 353 355.Z. · Zbl 0276.62010
[6] CLARKE, B. S. and BARRON, A. R. 1990. Information-theoretic asy mptotics of Bay es methods. IEEE Trans. Inform. Theory 36 453 471. Z.
[7] CSISZAR, I. 1984. Sanov property, generalized I-projection and a conditional limit theorem. Ánn. Probab. 12 768 793. Z. · Zbl 0544.60011 · doi:10.1214/aop/1176993227
[8] DIACONIS, P. and FREEDMAN, D. 1986a. On the consistency of Bay es estimates. Ann. Statist. 14 1 26. Z. · Zbl 1006.60095 · doi:10.1023/A:1015170205728
[9] DIACONIS, P. and FREEDMAN, D. 1986b. On inconsistent Bay es estimates of location. Ann. Statist. 14 68 87. Z. · Zbl 1006.60095 · doi:10.1023/A:1015170205728
[10] DIACONIS, P. and FREEDMAN, D. 1993. Nonparametric binary regression: a Bayesian approach. Ann. Statist. 21 2108 2137. Z. Z. · Zbl 0797.62031 · doi:10.1214/aos/1176349413
[11] DOSS, H. 1985. Bayesian nonparametric estimation of the median II : asy mptotic properties of the estimates. Ann. Statist. 13 1445 1464. Z. · Zbl 0587.62071 · doi:10.1214/aos/1176349747
[12] ESCOBAR, M. D. 1994. Estimating normal means with a Dirichlet process prior. J. Amer. Statist. Assoc. 89 268 277. Z. JSTOR: · Zbl 0791.62039 · doi:10.2307/2291223
[13] ESCOBAR, M. D. and WEST, M. 1995. Bayesian density estimation and inference using mixtures. J. Amer. Statist. Assoc. 90 577 588. Z. JSTOR: · Zbl 0826.62021 · doi:10.2307/2291069
[14] ESCOBAR, M. D. and WEST, M. 1998. Computing nonparametric hierarchical models. Practical Nonparametric and Semiparametric Bayesian Statistics. Lecture Notes in Statist. 133 1 22. Springer, Berlin. Z. · Zbl 0918.62028
[15] FERGUSON, T. S. 1973. A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209 230. Z. · Zbl 0255.62037 · doi:10.1214/aos/1176342360
[16] FREEDMAN, D. and DIACONIS, P. 1983. On inconsistent Bay es estimates in the discrete case. Ann. Statist. 11 1109 1118. Z. · Zbl 0539.62003
[17] GHOSHAL, S., GHOSH, J. K. and RAMAMOORTHI, R. V. 1997a. Consistency issues in Bayesian nonparametrics. Technical report, Michigan State Univ. · Zbl 0984.81122 · doi:10.1016/S0550-3213(00)00377-1
[18] GHOSHAL, S., GHOSH, J. K. and RAMAMOORTHI, R. V. 1997b. Posterior consistency of Dirichlet mixtures in density estimation. Technical report, Michigan State Univ. Z. · Zbl 0984.81122 · doi:10.1016/S0550-3213(00)00377-1
[19] HECKMAN, J. and SINGER, B. 1984. A method for minimizing the impact of distributional assumptions in econometric models for duration data. Econometrica 52 271 320. Z. JSTOR: · Zbl 0547.62077 · doi:10.2307/1911491
[20] HONORE, B. E. 1990. Simple estimation of a duration model with unobserved heterogeneity. Économetrica 58 453 473. Z. JSTOR: · Zbl 0727.62111 · doi:10.2307/2938211
[21] ISHWARAN, H. 1996a. Identifiability and rates of estimation for scale parameters in location mixture models. Ann. Statist. 24 1560 1571. Z. · Zbl 0867.62010 · doi:10.1214/aos/1032298284
[22] ISHWARAN, H. 1996b. Uniform rates of estimation in the semiparametric Weibull mixture model. Ann. Statist. 24 1572 1585. Z. · Zbl 0867.62011 · doi:10.1214/aos/1032298285
[23] LINDSAY, B. 1985. Using empirical partially Bay es inference for increased efficiency. Ann. Statist. 13 914 931. Z. · Zbl 0601.62044 · doi:10.1214/aos/1176349646
[24] LINDSAY, B., CLOGG, C. C. and GREGO, J. 1991. Semiparametric estimation in the Rasch model and related exponential response models, including a simple latent class model for item analysis. J. Amer. Statist. Assoc. 86 96 107. Z. JSTOR: · Zbl 0735.62107 · doi:10.2307/2289719
[25] MACEACHERN, S. N. 1994. Estimating normal means with a conjugate sty le Dirichlet process prior. Comm. Statist. Simulation Comput. 23 727 741. Z. · Zbl 0825.62053 · doi:10.1080/03610919408813196
[26] MACEACHERN, S. N. 1998. Computational methods for mixture of Dirichlet process models. Practical Nonparametric and Semiparametric Bayesian Statistics. Lecture Notes in Statist. 133 23 44. Springer, Berlin. Z. · Zbl 0918.62064
[27] MAULDIN, R. D., SUDDERTH, W. D. and WILLIAMS, S. C. 1992. Poly a trees and random distribuťions. Ann. Statist. 20 1203 1221. Z. · Zbl 0765.62006
[28] SCHWARTZ, L. 1965. On Bay es procedures. Z. Wahrsch. Verw. Gebiete 4 10 26. Z. · Zbl 0158.17606
[29] SHEN, X. 1995. On the properties of Bay es procedures in general parameter spaces. Unpublished manuscript, Ohio State Univ. Z.
[30] SHORACK, G. R. and WELLNER, J. A. 1996. Empirical Processes with Applications to Statistics. Wiley, New York. Z.
[31] WASSERMAN, L. 1998. Asy mptotic properties of nonparametric Bayesian procedures. Practical Nonparametric and Semiparametric Bayesian Statistics. Lecture Notes in Statist. 133 293 304. Springer, Berlin. · Zbl 0918.62045
[32] CLEVELAND, OHIO 44195 E-MAIL: ishwaran@bio.ri.ccf.org
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.