Computation and application of generalized linear mixed model derivatives using lme4. (English) Zbl 1496.62211

Summary: Maximum likelihood estimation of generalized linear mixed models (GLMMs) is difficult due to marginalization of the random effects. Derivative computations of a fitted GLMM’s likelihood are also difficult, especially because the derivatives are not by-products of popular estimation algorithms. In this paper, we first describe theoretical results related to GLMM derivatives along with a quadrature method to efficiently compute the derivatives, focusing on fitted lme4 models with a single clustering variable. We describe how psychometric results related to item response models are helpful for obtaining the derivatives, as well as for verifying the derivatives’ accuracies. We then provide a tutorial on the many possible uses of these derivatives, including robust standard errors, score tests of fixed effect parameters, and likelihood ratio tests of non-nested models. The derivative computation methods and applications described in the paper are all available in easily obtained R packages.


62P15 Applications of statistics to psychology
Full Text: DOI arXiv


[1] Barr, DJ; Levy, R.; Scheepers, C.; Tily, HJ, Random effects structure for confirmatory hypothesis testing: Keep it maximal, Journal of Memory and Language, 68, 255-278 (2013) · doi:10.1016/j.jml.2012.11.001
[2] Bates, D. (2021). Computational methods for mixed models. lme4 Package Vignette. Retrieved from https://cran.r-project.org/web/packages/lme4/vignettes/Theory.pdf
[3] Bates, D.; Mächler, M.; Bolker, B.; Walker, S., Fitting linear mixed-effects models using lme4, Journal of Statistical Software, 67, 1, 1-48 (2015) · doi:10.18637/jss.v067.i01
[4] Bauer, DJ; Curran, PJ, The integration of continuous and discrete latent variable models: Potential problems and promising opportunities, Psychological Methods, 9, 1, 3-29 (2004) · doi:10.1037/1082-989X.9.1.3
[5] Bock, RD; Lieberman, M., Fitting a response model for n dichotomously scored items, Psychometrika, 35, 179-198 (1970) · doi:10.1007/BF02291262
[6] Bürkner, P-C, Advanced Bayesian multilevel modeling with the R package brms, The R Journal, 10, 1, 395-411 (2018) · doi:10.32614/RJ-2018-017
[7] Cai, L., High-dimensional exploratory item factor analysis by a Metropolis-Hastings Robbins-Monro algorithm, Psychometrika, 75, 1, 33-57 (2010) · Zbl 1272.62113 · doi:10.1007/s11336-009-9136-x
[8] Cai, L., A two-tier full-information item factor analysis model with applications, Psychometrika, 75, 4, 581-612 (2010) · Zbl 1208.62183 · doi:10.1007/s11336-010-9178-0
[9] Chalmers, RP, mirt: A multidimensional item response theory package for the R environment, Journal of Statistical Software, 48, 6, 1-29 (2012) · doi:10.18637/jss.v048.i06
[10] De Boeck, P.; Bakker, M.; Zwitser, R.; Nivard, M.; Hofman, A.; Tuerlinckx, F.; Partchev, I., The estimation of item response models with the lmer function from the lme4 package in R, Journal of Statistical Software, 39, 12, 1-28 (2011) · doi:10.18637/jss.v039.i12
[11] De Boeck, P.; Wilson, M., Explanatory item response models: A generalized linear and nonlinear approach (2004), New York: Springer, New York · Zbl 1098.91002 · doi:10.1007/978-1-4757-3990-9
[12] Doran, H.; Bates, D.; Bliese, P.; Dowling, M., Estimating the multilevel Rasch model: With the lme4 package, Journal of Statistical Software, 20, 2, 1-18 (2007) · doi:10.18637/jss.v020.i02
[13] Embretson, SE; Reise, SP, Item response theory for psychologists (2000), Mahwah, NJ: Erlbaum Associates, Mahwah, NJ
[14] Engle, R. F. (1984). Wald, likelihood ratio, and Lagrange multiplier tests in econometrics. In Z. Griliches & M. D. Intriligator (Eds.), Handbook of econometrics. Elsevier. · Zbl 0581.62094
[15] Fokkema, M., Smits, N., Zeileis, A., Hothorn, T., & Kelderman, H. (2018). Detecting treatment-subgroup interactions in clustered data with generalized linear mixed-effects model trees. Behavior Research Methods, 50, 2016-2034. Retrieved from http://link.springer.com/article/10.3758/s13428-017-0971-x
[16] Glas, CAW, A Rasch model with a multivariate distribution of ability, Objective measurement: Theory into practice, 1, 236-258 (1992)
[17] Glas, CAW, Detection of differential item functioning using Lagrange multiplier tests, Statistica Sinica, 8, 3, 647-667 (1998) · Zbl 0905.62114
[18] Glas, CAW, Modification indices for the 2-PL and the nominal response model, Psychometrika, 64, 273-294 (1999) · Zbl 1291.62207 · doi:10.1007/BF02294296
[19] Hothorn, T., & Zeileis, A. (2015). partykit: A modular toolkit for recursive partytioning in R. Journal of Machine Learning Research, 16, 3905-3909. Retrieved from http://jmlr.org/papers/v16/hothorn15a.html · Zbl 1351.62005
[20] Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability (Vol. 1, pp. 221-233). · Zbl 0212.21504
[21] Komboz, B.; Strobl, C.; Zeileis, A., Tree-based global model tests for polytomous Rasch models, Educational and Psychological Measurement, 78, 1, 128-166 (2018) · doi:10.1177/0013164416664394
[22] Liu, Q.; Pierce, DA, A note on Gauss-Hermite quadrature, Biometrika, 81, 624-629 (1994) · Zbl 0813.65053
[23] Lord, FM; Novick, MR, Statistical theories of mental test scores (1968), Reading, MA: Addison-Wesley, Reading, MA · Zbl 0186.53701
[24] Louis, T. A. (1982). Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 44(2), 226-233. · Zbl 0488.62018
[25] Matuschek, H.; Kliegl, R.; Vasishth, S.; Baayen, H.; Bates, D., Balancing Type I error and power in linear mixed models, Journal of Memory and Language, 94, 305-315 (2017) · doi:10.1016/j.jml.2017.01.001
[26] McCullagh, P.; Nelder, JA, Generalized linear models (1989), Boca Raton, FL: Chapman & Hall, Boca Raton, FL · Zbl 0588.62104 · doi:10.1007/978-1-4899-3242-6
[27] McCulloch, C. E., & Neuhaus, J. M. (2001). Generalized linear mixed models. Wiley. · Zbl 0964.62061
[28] McCulloch, C. E., & Neuhaus, J. M. (2005). Generalized linear mixed models. In P. Armitage & T. Colton (Eds.), Encyclopedia of Biostatistics. doi:10.1002/0470011815.b2a10021
[29] Merkle, EC; Fan, J.; Zeileis, A., Testing for measurement invariance with respect to an ordinal variable, Psychometrika, 79, 569-584 (2014) · Zbl 1303.62106 · doi:10.1007/s11336-013-9376-7
[30] Merkle, EC; Furr, D.; Rabe-Hesketh, S., Bayesian comparison of latent variable models: Conditional versus marginal likelihoods, Psychometrika, 84, 802-829 (2019) · Zbl 1431.62551 · doi:10.1007/s11336-019-09679-0
[31] Merkle, E. C., & You, D. (2018). nonnest2 : Tests of non-nested models [Computer software manual]. Retrieved from https://cran.r-project.org/package=nonnest2 (R package version 0.5- 2)
[32] Merkle, EC; You, D.; Preacher, KJ, Testing nonnested structural equation models, Psychological Methods, 21, 2, 151-163 (2016) · doi:10.1037/met0000038
[33] Merkle, EC; Zeileis, A., Tests of measurement invariance without subgroups: A generalization of classical methods, Psychometrika, 78, 59-82 (2013) · Zbl 1284.62733 · doi:10.1007/s11336-012-9302-4
[34] Naylor, JC; Smith, AFM, Applications of a method for the efficient computation of posterior distributions, Journal of the Royal Statistical Society C, 31, 214-225 (1982) · Zbl 0521.65017
[35] Nelder, J.; Lee, Y., Likelihood, quasi-likelihood and pseudolikelihood: some comparisons, Journal of the Royal Statistical Society: Series B (Methodological), 54, 1, 273-284 (1992)
[36] Open Source Psychometrics Project. (n.d.). Open psychology data: Raw data from online [Computer software manual]. Retrieved 2017-10-17, from https://openpsychometrics.org/_rawdata/
[37] Petersen, K. B., & Pedersen, M. S. (2012). The matrix cookbook. Technical University of Denmark. Retrieved from http://www2.imm.dtu.dk/pubdb/p.php3274 (Version 20121115)
[38] Pinheiro, JC; Bates, DM, Approximations to the log-likelihood function in the nonlinear mixed-effects model, Journal of Computational Graphics and Statistics, 4, 12-35 (1995)
[39] Powell, M. J. D. (2009). The BOBYQA algorithm for bound constrained optimization without derivatives. Report No. DAMTP 2009/NA06, Centre for Mathematical Sciences, University of Cambridge, UK. http://www.damtp.cam.ac.uk/user/na/NA_papers/NA2009_06.pdf.
[40] R Core Team. (2020). R: A language and environment for statistical computing [Computer software manual]. Vienna, Austria. Retrieved from https://www.R-project.org/
[41] Rabe-Hesketh, S.; Skrondal, A.; Pickles, A., Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects, Journal of Econometrics, 128, 2, 301-323 (2005) · Zbl 1336.62079 · doi:10.1016/j.jeconom.2004.08.017
[42] Rasbash, J.; Goldstein, H., Efficient analysis of mixed hierarchical and cross-classified random structures using a multilevel model, Journal of Educational and Behavioral Statistics, 19, 337-350 (1994) · doi:10.3102/10769986019004337
[43] Schabenberger, O. (2005). Introducing the GLIMMIX procedure for generalized linear mixed models. SUGI 30 Proceedings, 196.
[44] Schneider, L.; Chalmers, RP; Debelak, R.; Merkle, EC, Model selection of nested and non-nested item response models using Vuong tests, Multivariate Behavioral Research, 55, 664-684 (2020) · doi:10.1080/00273171.2019.1664280
[45] Shao, X.; Zhang, X., Testing for change points in time series, Journal of the American Statistical Association, 105, 491, 1228-1240 (2010) · Zbl 1390.62184 · doi:10.1198/jasa.2010.tm10103
[46] Skrondal, A.; Rabe-Hesketh, S., Generalized latent variable modeling: Multilevel, longitudinal, and structural equation modeling (2004), Boca Raton, FL: Chapman & Hall, Boca Raton, FL · Zbl 1097.62001 · doi:10.1201/9780203489437
[47] Strobl, C.; Kopf, J.; Zeileis, A., Rasch trees: A new method for detecting differential item functioning in the Rasch model, Psychometrika, 80, 2, 289-316 (2015) · Zbl 1322.62341 · doi:10.1007/s11336-013-9388-3
[48] Stroup, WW, Generalized linear mixed models: Modern concepts, methods and applications (2012), USA: CRC Press, USA
[49] Stroup, W. W., & Claassen, E. (2020). Pseudo-likelihood or quadrature What we thought we knew, what we think we know, and what we are still trying to figure out. Journal of Agricultural. Biological and Environmental Statistics, 25(4), 639-656. · Zbl 07603047
[50] Thall, P. F., & Vail, S. C. (1990). Some covariance models for longitudinal count data with overdispersion. Biometrics, 46(3), 657-671. · Zbl 0712.62048
[51] Trepte, S., & Verbeet, M. (Eds.). (2010). Allgemeinbildung in Deutschland-erkenntnisse aus dem SPIEGEL Studentenpisa-Test. Wiesbaden: VS Verlag.
[52] Vuong, QH, Likelihood ratio tests for model selection and non-nested hypotheses, Econometrica, 57, 307-333 (1989) · Zbl 0701.62106 · doi:10.2307/1912557
[53] Wang, T.; Merkle, EC, merDeriv: Derivative computations for linear mixed effects models with application to robust standard errors, Journal of Statistical Software, 87, 1, 1-16 (2018) · doi:10.18637/jss.v087.c01
[54] Wang, T., Merkle, E. C., Anguera, J. A., & Turner, B. M. (2020). Score-based tests for detecting heterogeneity in linear mixed models. Behavior Research Methods, 53, 216-231.
[55] Wang, T.; Strobl, C.; Zeileis, A.; Merkle, EC, Score-based tests of differential item functioning via pairwise maximum likelihood estimation, Psychometrika, 83, 1, 132-155 (2018) · Zbl 1402.62330 · doi:10.1007/s11336-017-9591-8
[56] White, H., A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity, Econometrica: Journal of the Econometric Society, 48, 4, 817-838 (1980) · Zbl 0459.62051 · doi:10.2307/1912934
[57] Zeileis, A. (2004). Econometric computing with HC and HAC covariance matrix estimators. Journal of Statistical Software, 11(10), 1-17. Retrieved from http://www.jstatsoft.org/v11/i10/
[58] Zeileis, A., Object-oriented computation of sandwich estimators, Journal of Statistical Software, 16, 9, 1-16 (2006) · doi:10.18637/jss.v016.i09
[59] Zeileis, A.; Hornik, K., Generalized M-fluctuation tests for parameter instability, Statistica Neerlandica, 61, 488-508 (2007) · Zbl 1152.62014 · doi:10.1111/j.1467-9574.2007.00371.x
[60] Zeileis, A., & Hothorn, T. (2002). Diagnostic checking in regression relationships. R News, 2(3), 7-10. Retrieved from https://CRAN.R-project.org/doc/Rnews/
[61] Zeileis, A., Köll, S., & Graham, N. (2020). Various versatile variances: An object-oriented implementation of clustered covariances in R. Journal of Statistical Software. doi:10.18637/jss.v095.i01
[62] Zeileis, A., Leisch, F., Hornik, K., & Kleiber, C. (2002). strucchange: An R package for testing for structural change in linear regression models. Journal of Statistical Software, 7(2), 1-38. Retrieved from http://www.jstatsoft.org/v07/i02/
[63] Zhang, X.; Shao, X.; Hayhoe, K.; Wuebbles, DJ, Testing the structural stability of temporally dependent functional observations and application to climate projections, Electronic Journal of Statistics, 5, 1765-1796 (2011) · Zbl 1271.62097
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.