×

zbMATH — the first resource for mathematics

A general framework for the parametrization of hierarchical models. (English) Zbl 1246.62195
Summary: We describe centering and noncentering methodology as complementary techniques for use in parametrization of broad classes of hierarchical models, with a view to the construction of effective MCMC algorithms for exploring posterior distributions from these models. We give a clear qualitative understanding as to when centering and noncentering work well, and introduce a theory concerning the convergence time complexity of Gibbs samplers using centered and noncentered parametrizations. We give general recipes for the construction of noncentered parametrizations, including an auxiliary variable technique called the state-space expansion technique. We also describe partially noncentered methods, and demonstrate their use in constructing robust Gibbs sampler algorithms whose convergence properties are not overly sensitive to the data.

MSC:
62M99 Inference from stochastic processes
65C40 Numerical analysis or methods applied to Markov chains
62F15 Bayesian inference
PDF BibTeX XML Cite
Full Text: DOI Euclid
References:
[1] Amit, Y. (1991). On rates of convergence of stochastic relaxation for Gaussian and non-Gaussian distributions. J. Multivariate Anal. 38 82–99. · Zbl 0735.60036
[2] Beskos, A., Papaspiliopoulos, O., Roberts, G. O. and Fearnhead, P. (2006). Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 68 333–382. · Zbl 1100.62079
[3] Carter, C. K. and Kohn, R. (1994). On Gibbs sampling for state space models. Biometrika 81 541–553. JSTOR: · Zbl 0809.62087
[4] Catlin, S. N., Abkowitz, J. L. and Guttorp, P. (2001). Statistical inference in a two-compartment model for hematopoiesis. Biometrics 57 546–553. JSTOR: · Zbl 1209.62264
[5] Chib, S., Pitt, M. and Shephard, N. (2004). Likelihood based inference for diffusion driven models. Working paper. Available at www.nuff.ox.ac.uk/economics/papers/2004/w20/chibpittshephard.pdf.
[6] Christensen, O. F., Roberts, G. O. and Sköld, M. (2006). Robust Markov chain Monte Carlo methods for spatial generalized linear mixed models J. Comput. Graph. Statist. 15 1–17.
[7] Christensen, O. F. and Waagepetersen, R. (2002). Bayesian prediction of spatial count data using generalized linear mixed models. Biometrics 58 280–286. JSTOR: · Zbl 1209.62156
[8] Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. J. Roy. Statist. Soc. Ser. B 56 363–375. JSTOR: · Zbl 0796.62028
[9] Diggle, P. J., Tawn, J. A. and Moyeed, R. A. (1998). Model-based geostatistics (with discussion). Appl. Statist. 47 299–350. JSTOR: · Zbl 0904.62119
[10] Escobar, M. D. (1994). Estimating normal means with a Dirichlet process prior. J. Amer. Statist. Assoc. 89 268–277. JSTOR: · Zbl 0791.62039
[11] Ferguson, T. S. (1974). Prior distributions on spaces of probability measures. Ann. Statist. 2 615–629. · Zbl 0286.62008
[12] Gelfand, A. E. (2000). Gibbs sampling. J. Amer. Statist. Assoc. 95 1300–1304. JSTOR: · Zbl 1072.65503
[13] Gelfand, A. E., Sahu, S. K. and Carlin, B. P. (1995). Efficient parameterisations for normal linear mixed models. Biometrika 82 479–488. JSTOR: · Zbl 0832.62064
[14] Gelfand, A. E., Sahu, S. K. and Carlin, B. P. (1996). Efficient parametrizations for generalized linear mixed models. In Bayesian Statistics 5 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 165–180. Oxford Univ. Press, New York.
[15] Ghysels, E., Harvey, A. C. and Renault, E. (1996). Stochastic volatility. In Statistical Methods in Finance (G. S. Maddala and C. R. Rao, eds.) 119–191. North-Holland, Amsterdam.
[16] Griffin, J. E. and Steel, M. F. J. (2003). Flexible mixture modelling of stochastic frontiers. Technical Report 418, Dept. Statistics, Univ. Warwick.
[17] Griffin, J. E. and Steel, M. F. J. (2004). Semiparametric Bayesian inference for stochastic frontier models. J. Econometrics 123 121–152. · Zbl 1328.62208
[18] Ishwaran, H. and Zarepour, M. (2000). Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models. Biometrika 87 371–390. JSTOR: · Zbl 0949.62037
[19] Jones, G. L. and Hobert, J. P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo. Statist. Sci. 16 312–334. · Zbl 1127.60309
[20] Kitagawa, G. (1987). Non-Gaussian state-space modeling of nonstationary time series (with discussion). J. Amer. Statist. Assoc. 82 1032–1063. JSTOR: · Zbl 0644.62088
[21] Knorr-Held, L. and Rue, H. (2002). On block updating in Markov random field models for disease mapping. Scand. J. Statist. 29 597–614. · Zbl 1039.62092
[22] Lavine, M. and West, M. (1992). A Bayesian method for classification and discrimination. Canad. J. Statist. 20 451–461. JSTOR: · Zbl 0765.62062
[23] Liu, J. S. (1994). Fraction of missing information and convergence rate of data augmentation. In Computing Science and Statistics : Proc. 26th Symposiom on the Interface 490–496. Interface Foundation of North America, Fairfax Station, VA.
[24] Liu, J. S., Wong, W. H. and Kong, A. (1995). Covariance structure and convergence rate of the Gibbs sampler with various scans. J. Roy. Statist. Soc. Ser. B 57 157–169. JSTOR: · Zbl 0811.60056
[25] Møller, J., Syversveen, A. R. and Waagepetersen, R. P. (1998). Log Gaussian Cox processes. Scand. J. Statist. 25 451–482. · Zbl 0931.60038
[26] Müller, P., Erkanli, A. and West, M. (1996). Bayesian curve fitting using multivariate normal mixtures. Biometrika 83 67–79. JSTOR: · Zbl 0865.62029
[27] Neal, P. and Roberts, G. O. (2005). A case study in noncentering for data augmentention: Stochastic epidemics. Stat. Comput. 15 315–327.
[28] Nielsen, B. and Shephard, N. (2003). Likelihood analysis of a first-order autoregressive model with exponential innovations. J. Time Ser. Anal. 24 337–344. · Zbl 1050.62096
[29] O’Neill, P. D. and Roberts, G. O. (1999). Bayesian inference for partially observed stochastic epidemics. J. Roy. Statist. Soc. Ser. A 162 121–129.
[30] Papaspiliopoulos, O. (2003). Non-centered parametrizations for hierarchical models and data augmentation. Ph.D. dissertation, Dept. Mathematics and Statistics, Lancaster Univ.
[31] Papaspiliopoulos, O. and Roberts, G. (2004). Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models. Available at www.warwick.ac.uk/staff/O.Papaspiliopoulos/research.html. · Zbl 1437.62576
[32] Papaspiliopoulos, O. and Roberts, G. (2005). Stability of the Gibbs sampler for Bayesian hierarchical models. Submitted. · Zbl 1144.65007
[33] Papaspiliopoulos, O., Roberts, G. O., and Sköld, M. (2003). Non-centered parameterizations for hierarchical models and data augmentation (with discussion). In Bayesian Statistics 7 (J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West, eds.) 307–326. Oxford Univ. Press, New York.
[34] Peruggia, M., Van Zandt, T. and Chen, M. (2002). Was it a car or a cat I saw? An analysis of response times for word recognition. Case Studies in Bayesian Statistics VI . Lecture Notes in Statist. 167 319–334. Springer, New York.
[35] Renshaw, E. and Gibson, G. J. (1998). Can Markov chain Monte Carlo be usefully applied to stochastic processes with hidden birth times? Inverse Problems 14 1581–1606. · Zbl 0953.60061
[36] Roberts, G. O., Papaspiliopoulos, O. and Dellaportas, P. (2004). Bayesian inference for non-Gaussian Ornstein–Uhlenbeck stochastic volatility processes. J. R. Stat. Soc. Ser. B Stat. Methodol. 66 369–393. · Zbl 1062.62049
[37] Roberts, G. O. and Sahu, S. K. (1997). Updating schemes, correlation structure, blocking and parameterization for the Gibbs sampler. J. Roy. Statist. Soc. Ser. B 59 291–317. JSTOR: · Zbl 0886.62083
[38] Roberts, G. O. and Stramer, O. (2001). On inference for partially observed nonlinear diffusion models using the Metropolis–Hastings algorithm. Biometrika 88 603–621. JSTOR: · Zbl 0985.62066
[39] Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys . Wiley, New York. · Zbl 1070.62007
[40] Rue, H. (2001). Fast sampling of Gaussian Markov random fields. J. R. Stat. Soc. Ser. B Stat. Methodol. 63 325–338. JSTOR: · Zbl 0979.62075
[41] Rue, H. and Tjelmeland, H. (2002). Fitting Gaussian Markov random fields to Gaussian fields. Scand. J. Statist. 29 31–49. · Zbl 1017.62088
[42] Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statist. Sinica 4 639–650. · Zbl 0823.62007
[43] Shephard, N. (1994). Partial non-Gaussian state space. Biometrika 81 115–131. JSTOR: · Zbl 0796.62079
[44] Walker, S. G., Damien, P., Laud, P. W. and Smith, A. F. M. (1999). Bayesian nonparametric inference for random distributions and related functions (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 61 485–527. JSTOR: · Zbl 0983.62027
[45] White, J. S. (1958). The limiting distribution of the serial correlation coefficient in the explosive case. Ann. Math. Statist. 29 1188–1197. · Zbl 0099.13004
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.