×
Kyung, Minjung; Gill, Jeff; Casella, George

Estimation in Dirichlet random effects models. (English) Zbl 1183.62034

Ann. Stat. 38, No. 2, 979-1009 (2010).
Summary: We develop a new Gibbs sampler for a linear mixed model with a Dirichlet process random effect term, which is easily extended to a generalized linear mixed model with a probit link function. Our Gibbs sampler exploits the properties of the multinomial and Dirichlet distributions, and is shown to be an improvement, in terms of operator norm and efficiency, over other commonly used MCMC algorithms. We also investigate methods for the estimation of the precision parameter of the Dirichlet process, finding that maximum likelihood may not be desirable, but a posterior mode is a reasonable approach. Examples are given to show how these models perform on real data. Our results complement both the theoretical basis of the Dirichlet process nonparametric prior and the computational work that has been done to date.

Cited in 17 Documents

MSC:

62F10 Point estimation
62J12 Generalized linear models (logistic models)
62F15 Bayesian inference
65C60 Computational problems in statistics (MSC2010)
62G05 Nonparametric estimation
62F99 Parametric inference
62P25 Applications of statistics to social sciences
62G99 Nonparametric inference

Keywords:

linear mixed models; generalized linear mixed models; hierarchical models; Gibbs sampling; Bayes estimation
PDF BibTeX XML Cite
\textit{M. Kyung} et al., Ann. Stat. 38, No. 2, 979--1009 (2010; Zbl 1183.62034)
Full Text: DOI arXiv

References:

[1] Andrews, D. F. and Mallows, C. L. (1974). Scale mixtures of normal distributions. J. Roy. Statist. Soc. Ser. B 36 99-102. JSTOR: · Zbl 0282.62017
[2] Albert, J. H. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. J. Amer. Statist. Assoc. 88 669-679. JSTOR: · Zbl 0774.62031
[3] Barry, D. and Hartigan, J. A. (1992). Product partition models for change point problems. Ann. Statist. 20 260-279. · Zbl 0780.62071
[4] Blackwell, D. and MacQueen, J. B. (1973). Ferguson distributions via Pólya urn schemes. Ann. Statist. 1 353-355. · Zbl 0276.62010
[5] Booth, J. G., Casella, G. and Hobert, J. P. (2008). Clustering using objective functions and stochastic search. J. Roy. Statist. Soc. Ser. B 70 119-140. · Zbl 1400.62128
[6] Breslow, N. E. and Clayton, D. G. (1993). Approximate inference in generalized linear mixed models. J. Amer. Statist. Assoc. 88 9-25. · Zbl 0775.62195
[7] Burr, D. and Doss, H. (2005). A Bayesian semi-parametric model for random effects meta-analysis. J. Amer. Statist. Assoc. 100 242-251. · Zbl 1117.62304
[8] Casella, G. (2001). Empirical Bayes Gibbs sampling. Biostatistics 2 485-500. · Zbl 0891.65016
[9] Crowley, E. M. (1997). Product partition models for normal means. J. Amer. Statist. Assoc. 92 192-198. · Zbl 0889.62011
[10] Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1-67. · Zbl 0595.62022
[11] Dorazio, R. M., Mukherjee, B., Zhang, L., Ghosh, M., Jelks, H. L. and Jordan, F. (2008). Modelling unobserved sources of heterogeneity in animal abundance using a Dirichlet process prior. Biometrics 64 635-644. · Zbl 1137.62084
[12] Doss, H. (1985a). Bayesian nonparametric estimation of the median. I: Computation of the estimates. Ann. Statist. 13 1432-1444. · Zbl 0587.62070
[13] Doss, H. (1985b). Bayesian nonparametric estimation of the median. II: Asymptotic properties of the estimates. Ann. Statist. 13 1445-1464. · Zbl 0587.62071
[14] Doss, H. (1994). Bayesian nonparametric estimation for incomplete data via successive substitution sampling. Ann. Statist. 22 1763-1786. · Zbl 0824.62027
[15] Doss, H. (2008). Estimation of Bayes factors for nonparametric Bayes problems via Radon-Nikodym derivatives. Technical report, Dept. Statistics, Univ. Florida.
[16] Escobar, M. D. and West, M. (1995). Bayesian density estimation and inference using mixtures. J. Amer. Statist. Assoc. 90 577-588. JSTOR: · Zbl 0826.62021
[17] Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209-230. · Zbl 0255.62037
[18] Ghosh, M., Natarajan, K., Stroud, T. W. F. and Carlin, B. P. (1998). Generalized linear models for small-area estimation. J. Amer. Statist. Assoc. 93 273-282. JSTOR: · Zbl 0906.62068
[19] Ghosal, S. (2009). Dirichlet process, related priors and posterior asymptotics. In Bayesian Nonparametrics in Practice (N. L. Hjort et al., eds.). Cambridge Univ. Press.
[20] Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999). Consistent semiparametric Bayesian inference about a location parameter. J. Statist. Plann. Inference 77 181-193. · Zbl 1054.62528
[21] Gill, J. and Casella, G. (2009). Nonparametric priors for ordinal Bayesian social science models: Specification and estimation. J. Amer. Statist. Assoc. 104 453-464. · Zbl 1388.62377
[22] Hartigan, J. A. (1990). Partition models. Comm. Statist. 19 2745-2756. · Zbl 04500491
[23] Hobert, J. P. and Marchev, D. (2008). A theoretical comparison of the data augmentation, marginal augmentation and PX-DA algorithms. Ann. Statist. 36 532-554. · Zbl 1155.60031
[24] Korwar, R. M. and Hollander, M. (1973). Contributions to the theory of Dirichlet processes. Ann. Probab. 1 705-711. · Zbl 0264.60084
[25] Kyung, M., Gill, J. and Casella, G. (2009). Sampling schemes for generalized linear Dirichlet random effects models. Technical report, Dept. Statistics, Univ. Florida. Available at www.stat.ufl.edu/ casella/Papers. · Zbl 1241.65007
[26] Liu, J. S. (1996). Nonparametric hierarchical Bayes via sequential imputations. Ann. Statist. 24 911-930. · Zbl 0880.62038
[27] Lo, A. Y. (1984). On a class of Bayesian nonparametric estimates: I. Density estimates. Ann. Statist. 12 351-357. · Zbl 0557.62036
[28] MacEachern, S. N. and Müller, P. (1998). Estimating mixture of Dirichlet process models. J. Comput. Graph. Statist. 7 223-238.
[29] McCullagh, P. and Yang, J. (2006). Stochastic classification models. In International Congress of Mathematicians III 669-686. Eur. Math. Soc., Zürich. · Zbl 1112.62058
[30] Mira, A. (2001). Ordering and improving the performance of Monte Carlo Markov chains. Statist. Sci. 16 340-350. · Zbl 1127.60312
[31] Mira, A. and Geyer, C. J. (1999). Ordering Monte Carlo Markov chains. Technical Report 632, School of Statistics, Univ. Minnesota.
[32] Mukhopadhyay, S. and Gelfand, A. E. (1997). Dirichlet process mixed generalized linear models. J. Amer. Statist. Assoc. 92 633-679. JSTOR: · Zbl 0889.62062
[33] Neal, R. M. (2000). Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Graph. Statist. 9 249-265. JSTOR:
[34] Naskar, M. and Das, K. (2004). Inference in Dirichlet process mixed generalized linear models by using Monte Carlo EM. Aust. N. Z. J. Stat. 46 685-701. · Zbl 1061.62107
[35] Naskar, M. and Das, K. (2006). Semiparametric analysis of two level bivariate binary data. Biometrics 62 1004-1013. · Zbl 1116.62034
[36] Pitman, J. (1996). Some developments of the Blackwell-MacQueen urn scheme. In Statistics, Probability and Game Theory (T. S. Ferguson, L. S. Shipley and J. B. MacQueen, eds.) 30 245-267. IMS, Hayward, CA.
[37] Quintana, F. A. and Iglesias, P. L. (2003). Bayesian clustering and product partition models. J. Roy. Statist. Soc. Ser. B 65 557-574. JSTOR: · Zbl 1065.62115
[38] Roy, V. and Hobert, J. P. (2007). Convergence rates and asymptotic standard errors for Markov chain Monte Carlo algorithms for Bayesian probit regression. J. Roy. Statist. Soc. Ser. B 69 607-623.
[39] Schwartz, L. (1965). On Bayes procedures. Probab. Theory Related Fields 4 10-46. · Zbl 0158.17606
[40] Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statist. Sinica 4 639-650. · Zbl 0823.62007
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.