×

Fast and accurate binary response mixed model analysis via expectation propagation. (English) Zbl 1452.62067

Summary: Expectation propagation is a general prescription for approximation of integrals in statistical inference problems. Its literature is mainly concerned with Bayesian inference scenarios. However, expectation propagation can also be used to approximate integrals arising in frequentist statistical inference. We focus on likelihood-based inference for binary response mixed models and show that fast and accurate quadrature-free inference can be realized for the probit link case with multivariate random effects and higher levels of nesting. The approach is supported by asymptotic calculations in which expectation propagation is seen to provide consistent estimation of the exact likelihood surface. Numerical studies reveal the availability of fast, highly accurate and scalable methodology for binary mixed model analysis.

MSC:

62-08 Computational methods for problems pertaining to statistics
62J12 Generalized linear models (logistic models)
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[2] Baayen, R.H., Davidson, D.J. and Bates, D.M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390-412.
[3] Bates, D., Maechler, M. and Bolker, B. (2014). mlmRev: Examples from multilevel modelling software review. R package version 1.0.
[4] http://cran.r-project.org.
[5] Baltagi, B.H. (2013). Econometric Analysis of Panel Data, Fifth Edition. Chichester, U.K.: John Wiley & Sons.
[6] Bates, D., Maechler, M., Bolker, B. and Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1-48.
[7] Bishop, C.M. (2006). Pattern Recognition and Machine Learning. New York: Springer. · Zbl 1107.68072
[8] Broyden, C.G. (1970). The convergence of a class of double-rank minimization algorithms. Journal of the Institute of Mathematics and Its Applications, 6, 76-90. · Zbl 0223.65023
[9] Carlin, B.P. and Gelfand, A.E. (1991). A sample reuse method for accurate parametric empirical Bayes confidence intervals. Journal of the Royal Statistical Society, Series B, 53, 189-200. · Zbl 0800.62183
[10] Dehaene, G. and Barthelmé, S. (2018). Expectation propagation in the large-data limit. Journal of the Royal Statistical Society, Series B, 80, 199-217. · Zbl 1380.62108
[11] Diggle, P., Heagerty, P., Liang, K.-L. and Zeger, S. (2002). Analysis of Longitudinal Data, Second Edition. Oxford, U.K.: Oxford University Press.
[12] Fletcher, R. (1970). A new approach to variable metric algorithms. Computer Journal, 13, 317-322. · Zbl 0207.17402
[13] Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B.,Vehtari, A. and Rubin, D.B. (2014). Bayesian Data Analysis, Third Edition, Boca Raton, Florida: CRC Press. · Zbl 1279.62004
[14] Gelman, A. and Hill, J. (2007). Data Analysis using Regression and Multilevel/Hierarchical Models. New York: Cambridge University Press.
[15] Givens, G.H. and Hoetig, J.A. (2005). Computational Statistics, Hoboken, New Jersey: John Wiley & Sons. · Zbl 1079.62001
[16] Goldfarb, D. (1970). A family of variable metric updates derived by variational means. Mathematics of Computation, 24, 23-26. · Zbl 0196.18002
[17] Goldstein, H. (2010). Multilevel Statistical Models, Fourth Edition. Chichester, U.K.: John Wiley & Sons.
[18] Harville, D.A. (2008). Matrix Algebra from a Statistician’s Perspective. New York: Springer. · Zbl 1142.15001
[19] Huang, A. and Wand, M.P. (2013). Simple marginally noninformative prior distributions for covariance matrices. Bayesian Analysis, 8, 439-452. · Zbl 1329.62135
[20] Jeon, M., Rijmen, F. and Rabe-Hesketh, S. (2017). A variational maximization-maximization algorithm for generalized linear mixed models with crossed random effects. Psychometrika, 82, 693-716. · Zbl 1402.62043
[21] Jiang, J. (2017). Asymptotic Analysis of Mixed Effects Models. Boca Raton, Florida: CRC Press. · Zbl 1387.62004
[22] Kim, A.S.I. and Wand, M.P. (2018). On expectation propagation for generalised, linear and mixed models. Australian and New Zealand Journal of Statistics, 60, 75-102. · Zbl 1521.62091
[23] Lele, S.R., Nadeem, K. and Schmuland, B. (2010). Estimability and likelihood inference for generalized linear mixed models using data cloning. Journal of the American Statistical Association, 105, 1617-1625. · Zbl 1388.62220
[24] Magnus, J.R. and Neudecker, H. (1999). Matrix Differential Calculus with Applications in Statistics and Econometrics, Revised Edition. Chichester U.K.: Wiley. · Zbl 0912.15003
[25] Mardia, K.V., Kent, J.T. and Bibby, J.M. (1979). Multivariate Analysis. London: Academic Press. · Zbl 0432.62029
[26] The Mathworks Incorporated (2018). Natick, Massachusetts, U.S.A.
[27] McCullagh, P. and Nelder, J.A. (1989). Generalized Linear Models, Second Edition. London: Chapman and Hall. · Zbl 0744.62098
[28] McCulloch, C.E., Searle, S.R. and Neuhaus, J.M. (2008). Generalized, Linear, and Mixed Models, Second Edition. New York: John Wiley & Sons. · Zbl 1165.62050
[29] Minka, T.P. (2001). Expectation propagation for approximate Bayesian inference. In J.S. Breese & D. Koller (eds), Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pp. 362-369. Burlington, Massachusetts: Morgan Kaufmann.
[30] Minka, T. (2005). Divergence measures and message passing. Microsoft Research Technical Report Series, MSR-TR-2005-173, 1-17.
[31] Minka, T. & Winn, J. (2008). Gates: A graphical notation for mixture models. Microsoft Research Technical Report Series, MSR-TR-2008-185, 1-16.
[32] Nelder, J.A. and Mead, R. (1965). A simplex method for function minimization. Computer Journal, 7, 308-313. · Zbl 0229.65053
[33] Ogden, H.E. (2015). A sequential reduction method for inference in generalized linear mixed models. Electronic Journal of Statistics, 9, 135-152. · Zbl 1307.62057
[34] Pinheiro, J.C. and Bates, D.M. (2000). Mixed-Effects Models in S and S-PLUS. New York: Springer. · Zbl 0953.62065
[36] Rao, J.N.K. and Molina, I. (2015). Small Area Estimation, Second Edition. Hoboken, New Jersey: John Wiley & Sons. · Zbl 1323.62002
[37] Rodríguez, G. and Goldman, N. (1995). An assessment of estimation procedures for multilevel models with binary responses. Journal of the Royal Statistical Society, Series A, 158, 73-89.
[38] Rue, H., Martino, S. and Chopin, N. (2009). Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion). Journal of the Royal Statistical Society, Series B, 71, 319-392. · Zbl 1248.62156
[39] Shanno, D.F. (1970). Conditioning of quasi-Newton methods for function minimization. Mathematics of Computation, 24, 647-656. · Zbl 0225.65073
[40] Sólymos, P. (2010). dclone: data cloning in R. The R Journal, 2/2, 29-37.
[42] The Mathworks Incorporated (2018). Natick, Massachusetts, U.S.A.
[43] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58, 267-288. · Zbl 0850.62538
[44] Wainwright, M.J. and Jordan, M.I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1, 1-305. · Zbl 1193.62107
[45] Wand, M.P. and Ormerod, J.T. (2012). Continued fraction enhancement of Bayesian computing. Stat, 1, 31-41.
[47] Yu, J.C.F. (2020). Fast and Accurate Frequentist Generalized Linear Mixed Model Analysis via Expectation Propagation. Doctor of Philosophy thesis, University of Technology Sydney.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.