Implementation and performance issues in the Bayesian and likelihood fitting of multilevel models. (English) Zbl 1037.62013

The paper presents results of a large simulation study directed to comparison of (i) Bayesian and likelihood fitting methods, and (ii) computational efficiency of several estimation procedures based on Markov Chain Monte Carlo (MCMC) for several types of multilevel models, namely random slopes regression (RSR) and random-effects logistic (RELF) models. Being the co-developers of the Bayesian MCMC capabilities in the multilevel modelling package MlwiN, the authors examine the relative performance, in the sense of point and interval estimation accuracy, of likelihood and Bayesian fitting procedures in two-level RSR. In the case of RELF models they compare (in terms of required CPU time to achieve given accuracy) a number of MCMC fitting methods, including adaptive rejection sampling and a specially for MLwiN developed procedure based on adaptive hybrid Metropolis-Gibbs sampling.
It is demonstrated that the Bayesian approach with a particular choice of diffuse inverse Wishart prior distribution for (co)variance parameters performs at least as well as maximum likelihood in RSR models with medium sample sizes, but neither approach performs sufficiently well for small samples. An adaptive hybrid Metropolis-Gibbs sampling method outperforms the Gibbs sampling in RELF models, which have been considered.


62F15 Bayesian inference
65C40 Numerical analysis or methods applied to Markov chains
62J99 Linear inference, regression
65C60 Computational problems in statistics (MSC2010)


Full Text: DOI


[1] Breslow, N.E. and Clayton, D.G. (1993). Approximate inference in generalized linear mixed models. Journal of the American Statistical Association, 88, 9-25. · Zbl 0775.62195
[2] Brooks, S.P. and Draper, D. (2000). Comparing the efficiency of MCMC samplers. Technical report, Department of Mathematical Sciences, University of Bath, UK.
[3] Browne, W.J. (1998). Applying MCMC Methods to Multilevel Models. PhD dissertation, Department of Mathematical Sciences, University of Bath, UK.
[4] Browne, W.J. and Draper, D. (1999). A comparison of Bayesian and likelihood methods for fitting multilevel models. Submitted. · Zbl 1331.62126
[5] Bryk, A.S. and Raudenbush, S.W. (1992). Hierarchical Linear Models: Applications and Data Analysis Methods. London: Sage.
[6] Bryk, A.S., Raudenbush, S.W., Seltzer, M. and Congdon, R. (1988). An Introduction to HLM: Computer Program and User’s Guide (Second Edition). Chicago: University of Chicago Department of Education.
[7] Carlin, B. (1992). Discussion of “Hierarchical models for combining information and for meta-analysis,” by Morris, C.N. and Normand, S.L. In Bayesian Statistics4, Bernardo, J.M., Berger, J.O., Dawid, A.P. and Smith, A.F.M. (eds.), 336-338. Oxford: Clarendon Press.
[8] Draper, D. (1995). Assessment and propagation of model uncertainty (with discussion). Journal of the Royal Statistical Society, Series B, 57, 45-97. · Zbl 0812.62001
[9] Draper, D. (2000). Bayesian Hierarchical Modeling. New York: Springer-Verlag, forthcoming.
[10] Gelman, A., Carlin, J.B., Stern, H.S. and Rubin, D.B. (1995). Bayesian Data Analysis. London: Chapman & Hall.
[11] Gelman, A.; Roberts, GO; Gilks, WR; Bernardo, JM (ed.); Berger, JO (ed.); Dawid, AP (ed.); Smith, AFM (ed.), Efficient Metropolis jumping rules, 599-607 (1995), Oxford
[12] Gelman, A. and Rubin, D.B. (1992). Inference from iterative simulation using multiple sequences (with discussion). Statistical Science, 7, 457-511. · Zbl 1386.65060
[13] Gilks, W.R., Richardson, S. and Spiegelhalter, D.J. (1996). Markov Chain Monte Carlo in Practice. London: Chapman & Hall. · Zbl 0832.00018
[14] Gilks, W.R. and Wild, P. (1992). Adaptive rejection sampling for Gibbs sampling. Applied Statistics, 41, 337-348. · Zbl 0825.62407
[15] Goldstein, H. (1986). Multilevel mixed linear model analysis using iterative generalised least squares. Biometrika, 73, 43-56. · Zbl 0587.62143
[16] Goldstein, H. (1989). Restricted unbiased iterative generalised least squares estimation. Biometrika, 76, 622-623. · Zbl 0677.62064
[17] Goldstein, H. (1995). Multilevel Statistical Models, Second Edition. London: Edward Arnold. · Zbl 1014.62126
[18] Heath, A., Yang, M. and Goldstein, H. (1996). Multilevel analysis of the changing relationship between class and party in Britain, 1964-1992. Quality and Quantity, 30, 389-404.
[19] Liu, J. and Hodges, J.S. (1999). Characterizing modes of the likelihood, restricted likelihood, and posterior for hierarchical models. Technical Report 99-011, Division of Biostatistics, University of Minnesota.
[20] Longford, N.T. (1987). A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects. Biometrika, 74, 817-827. · Zbl 0628.62075
[21] Madigan, D., Gavrin, J. and Raftery, A.E. (1995). Eliciting prior information to enhance the predictive performance of Bayesian graphical models. Communications in Statistics, Theory and Methods, 24, 2271-2292. · Zbl 0937.62576
[22] Mortimore, P., Sammons, P., Stoll, L., Lewis, D. and Ecob, R. (1988). School Matters. Wells: Open Books.
[23] Müller, P. (1993). A generic approach to posterior integration and Gibbs sampling. Technical Report, ISDS, Duke University, Durham NC.
[24] Pinheiro, J.C. and Bates, D.M. (1995). Approximations to the log-likelihood function in the non-linear mixed-effects model. Journal of Computational and Graphical Statistics, 4, 12-35.
[25] Raftery, AL; Lewis, S.; Bernardo, JM (ed.); Berger, JO (ed.); Dawid, AP (ed.); Smith, AFM (ed.), How many iterations in the Gibbs sampler, 763-774 (1992), Oxford
[26] Rasbash, J., Browne, W.J., Goldstein, H., Yang, M., Plewis, I., Draper, D., Healy, M. and Woodhouse, G. (1999). A User’s Guide to MLwiN, Version 2.0, London: Institute of Education, University of London.
[27] Raudenbush, S.W., Yang, M.-L. and Yosef, M. (2000). Maximum likelihood for hierarchical models via high-order multivariate Laplace approximations. Journal of Computational and Graphical Statistics, forthcoming.
[28] Ripley, B.D. (1987). Stochastic Simulation. New York: Wiley.
[29] Spiegelhalter, D.J., Thomas, A., Best, N.G. and Gilks, W.R. (1997). BUGS: Bayesian Inference Using Gibbs Sampling, Version 0.60. Cambridge: Medical Research Council Biostatistics Unit.
[30] Woodhouse, G., Rasbash, J., Goldstein, H., Yang, M., Howarth, J. and Plewis, I. (1995). A Guide toMLnfor New Users. London: Institute of Education, University of London.
[31] Zeger, S.L. and Karim, M.R. (1991). Generalized linear models with random effects: a Gibbs sampling approach. Journal of the American Statistical Association, 86, 79-86.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.