Testing random effects in linear mixed models: another look at the F-test (with discussion). (English) Zbl 1420.62294

Summary: This article re-examines the F-test based on linear combinations of the responses, or FLC test, for testing random effects in linear mixed models. In current statistical practice, the FLC test is underused and we argue that it should be reconsidered as a valuable method for use with linear mixed models. We present a new, more general derivation of the FLC test which applies to a broad class of linear mixed models where the random effects can be correlated. We highlight three advantages of the FLC test that are often overlooked in modern applications of linear mixed models, namely its computation speed, its generality, and its exactness as a test. Empirical studies provide new insight into the finite sample performance of the FLC test, identifying cases where it is competitive or even outperforms modern methods in terms of power, as well as settings in which it performs worse than simulation-based methods for testing random effects. In all circumstances, the FLC test is faster to compute.


62J05 Linear regression; mixed models
62F40 Bootstrap, jackknife and other resampling methods
Full Text: DOI


[1] Baksalary, J.K. & Kala, R. (1981). Linear transformations preserving best linear unbiased estimators in a general Gauss‐Markov model. Annals of Statistics9, 913- 916. · Zbl 0471.62067
[2] Baksalary, J.K. & Kala, R. (1986). Linear suffciency with respect to a given vector of parametric functions. Journal of Statistical Plannning and Inference14, 331- 338. · Zbl 0614.62079
[3] Bartlett, M.S. (1937). Properties of sufficiency and statistical tests. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences160, 268- 282. · Zbl 0016.41201
[4] Bates, D., Maechler, M. & Bolker, B. (2014). mlmRev: Examples from multilevel modelling software review. Available from URL: https://CRAN.R-project.org/package=mlmRev. R package version 1.0‐6.
[5] Bates, D., Maechler, M., Bolker, B. & Walker, S. (2014). lme4: Linear mixed‐effects models using Eigen and S4. Available from URL: http://CRAN.R-project.org/package=lme4. R package version 1.1‐7 [Last accessed 7 July 2017].
[6] Brien, C. (2016). asremlPlus. Available from URL: https://cran.r-project.org/web/packages/asremlPlus/index.html. R package version 2.0‐12 [Last accessed 7 July 2017].
[7] Butler, D.G., Cullis, B.R., Gilmour, A.R., Gogel, B.J. & Thompson, R. (2017). ASReml‐R Reference Manual, Version 4. Hemel Hempstead: VSN International.
[8] Cook, R.D. & Weisberg, S. (1982). Residuals and Influence in Regression. London: Chapman and Hall. · Zbl 0564.62054
[9] Crainiceanu, C.M. & Ruppert, D. (2004). Likelihood ratio tests in linear mixed models with one variance component. Journal of the Royal Statistical Society: Series B (Statistical Methodology)66, 165- 185. · Zbl 1061.62027
[10] Deming, W.E. (1975). On probability as a basis for action. The American Statistician29, 146- 152.
[11] Dodge, H.F. & Romig, H.G. (1998). Sampling Inspection Tables. New York: JohnWiley & Sons, 2nd edn. · Zbl 0893.62102
[12] Dodge, H.F. & Romig, H.G. (1929). A method of sampling inspection. Bell System Technical Journal8, 613- 631.
[13] Drikvandi, R., Verbeke, G., Khodadadi, A. & Nia, V.P. (2013). Testing multiple variance components in linear mixed‐effects models. Biostatistics14, 144- 159.
[14] Drygas, H. (1983). Sufficiency and completeness in the general Gauss Markov model. Sankhya45, 88- 98. · Zbl 0535.62007
[15] Eisenhart, C. (1947). The assumptions underlying the analysis of variance. Biometrics3, 1- 21.
[16] Fan, Y. & Li, R. (2012). Variable selection in linear mixed effects models. Annals of Statistics40, 2043- 2068. · Zbl 1257.62077
[17] Fan, Y. & Tang, C.Y. (2012). Tuning parameter selection in high dimensional penalized likelihood. Journal of the Royal Statistical Society: Series B (Statistical Methodology)75, 531- 552. · Zbl 1411.62216
[18] Fitzmaurice, G.M., Lipsitz, S.R. & Ibrahim, J.G. (2007). A note on permutation tests for variance components in multilevel generalized linear mixed models. Biometrics63, 942- 946. · Zbl 1146.62086
[19] Gelman, A. (2005). Analysis of variance: why it is more important than ever. The Annals of Statistics33, 1- 31. · Zbl 1064.62082
[20] Giampaoli, V. & Singer, J.M. (2009). Likelihood ratio tests for variance components in linear mixed models. Journal of Statistical Planning and Inference139, 1435- 1448. · Zbl 1419.62033
[21] Gilmour, A.R., Cullis, B.R. & Verbyla, A.P. (1997). Accounting for natural and extraneous variation in the analysis of field experiments. Journal of Agricultural, Biological, and Environmental Statistics2, 269- 293.
[22] Greven, S., Crainiceanu, C.M., Kuchenhoff, H. & Peters, A. (2008). Restricted likelihood ratio testing for zero variance components in linear mixed models. Journal of Computational and Graphical Statistics17, 870- 891.
[23] Halekoh, U. & H⊘jsgaard, S. (2014). A Kenward‐Roger approximation and parametric bootstrap methods for tests in linear mixed models - The R Package pbkrtest. Journal of Statistical Software59, 1- 30. URL: http://www.jstatsoft.org/v59/i09/.
[24] Haslett, S.J., Puntanen, S. & Arendacká, B. (2015). The link between the fixed and mixed linear models revisited. Statistical Papers56, 849- 861. Doi: 10:1007/s00362‐014‐0611‐9.
[25] Haslett, S.J., Liu, X.Q., Markiewicz, A. & Puntanen, S. (2017). Some properties of linear sufficiency and the BLUPs in the linear mixed model. Statistical Papers. Published online 25 August 2017; doi: 10.1007/s00362‐017‐0943‐3.
[26] Hodges, J.S. (2016). Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects. Boca Raton, FL: Chapman and Hall/CRC. · Zbl 1282.62197
[27] Hui, F.K.C., Müller, S. & Welsh, A.H. (2017). Joint selection in mixed models using regularized PQL. Journal of the American Statistical Association112, 1323- 1333.
[28] Hui, F.K.C., Müller, S. & Welsh, A.H. (2018). Sparse pairwise likelihood estimation for multivariate longitudinal mixed models. Journal of the American Statistical Association113, 1759- 1769. · Zbl 1409.62151
[29] Jiang, J. (1999). Conditional inference about generalized linear mixed models. The Annals of Statistics27, 1974- 2007.
[30] Jiang, J. (2007). Linear and Generalized Linear Mixed Models and Their Applications. New York: Springer‐Verlag. · Zbl 1152.62040
[31] Kenward, M.G. & Roger, J.H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics53, 983- 997. · Zbl 0890.62042
[32] Li, Z. (2015). A residual‐based test for variance components in linear mixed models. Statistics & Probability Letters98, 73- 78. · Zbl 1321.62088
[33] Li, Z., Chen, F. & Zhu, L. (2014). Variance components testing in ANOVA‐type mixed models. Scandinavian Journal of Statistics41, 482- 496. · Zbl 1416.62417
[34] Lin, T.H. & Harville, D.A. (1991). Some alternatives to Wald’s confidence interval and test. Journal of the American Statistical Association86, 179- 187.
[35] Lin, B., Pang, Z. & Jiang, J. (2013). Fixed and random effects selection by reml and pathwise coordinate optimization. Journal of Computational and Graphical Statistics22, 341- 355.
[36] Molenberghs, G. & Verbeke, G. (2007). Likelihood ratio, score, and wald tests in a constrained parameter space. The American Statistician61, 22- 27.
[37] Müller, S., Scealy, J.L., Welsh, A.H. (2013). Model selection in linear mixed models. Statistical Science28, 135- 167.
[38] Nobre, J.S., Singer, J.M. & Sen, P.K. (2013). U‐tests for variance components in linear mixed models. Test22, 580- 605. · Zbl 1283.62096
[39] Patterson, H.D. & Thompson, R. (1971). Recovery of inter‐block information when block sizes are unequal. Biometrika58, 545- 554. · Zbl 0228.62046
[40] Qu, L. (2015). varComp: Variance Component Models. Available from URL: http://CRAN.R-project.org/package=varComp. R package #version 0.1‐360 [Last accessed 7 July 2017].
[41] Qu, L., Guennel, T. & Marshall, S.L. (2013). Linear score tests for variance components in linear mixed models and applications to genetic association studies. Biometrics69, 883- 892. · Zbl 1419.62035
[42] Rao, J. & Molina, I. (2015). Small Area Estimation. New Jersery: Wiley. · Zbl 1323.62002
[43] Samuh, M.H., Grilli, L., Rampichini, C., Salmaso, L. & Lunardon, N. (2012). The use of permutation tests for variance components in linear mixed models. Communications in Statistics‐Theory and Methods41, 3020- 3029. · Zbl 1296.62103
[44] Satterthwaite, F.E. (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin2, 110- 114.
[45] Scheipl, F., Greven, S. & Kuechenhoff, H. (2008). Size and power of tests for a zero random effect variance or polynomial regression in additive and linear mixed models. Computational Statistics & Data Analysis52, 3283- 3299. · Zbl 1452.62531
[46] Seber, G. (2015). The Linear Model and Hypothesis: A General Unifying Theory. Springer Series in Statistics, Springer International Publishing. · Zbl 1371.62002
[47] Seely, J.F., El‐Bassiouni, Y. (1983). Applying Wald’s variance component test. The Annals of Statistics11, 197- 201. · Zbl 0516.62028
[48] Self, S.G. & Liang, K. (1987). Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. Journal of the American Statistical Association82, 605- 610. · Zbl 0639.62020
[49] Silvapulle, M.J. & Silvapulle, P. (1995). A score test against one‐sided alternatives. Journal of the American Statistical Association90, 342- 349. · Zbl 0818.62022
[50] Sinha, S.K. (2009). Bootstrap tests for variance components in generalized linear mixed models. Canadian Journal of Statistics37, 219- 234. · Zbl 1176.62013
[51] Stigler, S.M. (1980). Stigler’s law of eponymy. Transactions of the New York Academy of Sciences39, 147- 158.
[52] Stram, D.O. & Lee, J.W. (1994). Variance components testing in the longitudinal mixed effects model. Biometrics50, 1171- 1177. · Zbl 0826.62054
[53] Verbeke, G. & Molenberghs, G. (2003). The use of score tests for inference on variance components. Biometrics59, 254- 262. · Zbl 1210.62013
[54] Verbeke, G. & Molenberghs, G. (2009). Linear Mixed Models for Longitudinal Data. New York: Springer‐Verlag. · Zbl 1162.62070
[55] Wald, A. (1947). A note on regression analysis. The Annals of Mathematical Statistics18, 586- 589. · Zbl 0029.30703
[56] Wallis, W.A. (1980). The statistical research group, 1942‐1945. Journal of the American Statistical Association75, 320- 330.
[57] Yates, F. (1967). A fresh look at the basic principles of the design and analysis of experiments. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 4: Biology and Problems of Health, eds. L.M.L. Cam and J. Neyman. The Regents of the University of California, pp. 777- 790.
[58] Zwick, R. & Sklar, J.C. (2005). Predicting college grades and degree completion using high school grades and SAT scores: The role of student ethnicity and first language. American Educational Research Journal42, 439- 464.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.