Variable selection in linear mixed effects models. (English) Zbl 1257.62077

Summary: This paper is concerned with the selection and estimation of fixed and random effects in linear mixed effects models. We propose a class of nonconcave penalized profile likelihood methods for selecting and estimating important fixed effects. To overcome the difficulty of unknown covariance matrix of random effects, we propose to use a proxy matrix in the penalized profile likelihood. We establish conditions on the choice of the proxy matrix and show that the proposed procedure enjoys the model selection consistency where the number of fixed effects is allowed to grow exponentially with the sample size. We further propose a group variable selection strategy to simultaneously select and estimate important random effects, where the unknown covariance matrix of random effects is replaced with a proxy matrix. We prove that, with the proxy matrix appropriately chosen, the proposed procedure can identify all true random effects with asymptotic probability one, where the dimension of random effects vector is allowed to increase exponentially with the sample size. Monte Carlo simulation studies are conducted to examine the finite-sample performance of the proposed procedures. We further illustrate the proposed procedures via a real data example.


62J05 Linear regression; mixed models
62H12 Estimation in multivariate analysis
62J07 Ridge regression; shrinkage estimators (Lasso)
65C05 Monte Carlo methods


Full Text: DOI arXiv Euclid


[1] Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory ( Tsahkadsor , 1971) (B. N. Petrov and F. Csaki, eds.) 267-281. Akad. Kiadó, Budapest. · Zbl 0283.62006
[2] Bickel, P. J. and Levina, E. (2004). Some theory of Fisher’s linear discriminant function, “naive Bayes,” and some alternatives when there are many more variables than observations. Bernoulli 10 989-1010. · Zbl 1064.62073 · doi:10.3150/bj/1106314847
[3] Bondell, H. D., Krishna, A. and Ghosh, S. K. (2010). Joint variable selection for fixed and random effects in linear mixed-effects models. Biometrics 66 1069-1077. · Zbl 1233.62134 · doi:10.1111/j.1541-0420.2010.01391.x
[4] Box, G. E. P. and Tiao, G. C. (1973). Bayesian Inference in Statistical Analysis . Addison-Wesley, Reading, MA. · Zbl 0271.62044
[5] Chen, Z. and Dunson, D. B. (2003). Random effects selection in linear mixed models. Biometrics 59 762-769. · Zbl 1214.62027 · doi:10.1111/j.0006-341X.2003.00089.x
[6] Dudoit, S., Fridlyand, J. and Speed, T. P. (2002). Comparison of discrimination methods for the classification of tumors using gene expression data. J. Amer. Statist. Assoc. 97 77-87. · Zbl 1073.62576 · doi:10.1198/016214502753479248
[7] Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression (with discussion). Ann. Statist. 32 407-451. · Zbl 1091.62054 · doi:10.1214/009053604000000067
[8] Fan, J. and Fan, Y. (2008). High-dimensional classification using features annealed independence rules. Ann. Statist. 36 2605-2637. · Zbl 1360.62327 · doi:10.1214/07-AOS504
[9] Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. · Zbl 1073.62547 · doi:10.1198/016214501753382273
[10] Fan, Y. and Li, R. (2012). Supplement to “Variable selection in linear mixed effects models.” . · Zbl 1257.62077
[11] Fan, J. and Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32 928-961. · Zbl 1092.62031 · doi:10.1214/009053604000000256
[12] Gelman, A., Carlin, J. B., Stern, H. S. and Rubin, D. B. (1995). Bayesian Data Analysis . Chapman & Hall, London. · Zbl 1279.62004
[13] Huang, J. Z., Wu, C. O. and Zhou, L. (2002). Varying-coefficient models and basis function approximations for the analysis of repeated measurements. Biometrika 89 111-128. · Zbl 0998.62024 · doi:10.1093/biomet/89.1.111
[14] Ibrahim, J. G., Zhu, H., Garcia, R. I. and Guo, R. (2011). Fixed and random effects selection in mixed effects models. Biometrics 67 495-503. · Zbl 1217.62171 · doi:10.1111/j.1541-0420.2010.01463.x
[15] Kaslow, R. A., Ostrow, D. G., Detels, R., Phair, J. P., Polk, B. F. and Rinaldo, C. R. (1987). The multicenter AIDS cohort study: Rationale, organization and selected characteristics of the participants. American Journal Epidemiology 126 310-318.
[16] Laird, N. M. and Ware, J. H. (1982). Random-effects models for longitudinal data. Biometrics 38 963-974. · Zbl 0512.62107 · doi:10.2307/2529876
[17] Liang, H., Wu, H. and Zou, G. (2008). A note on conditional AIC for linear mixed-effects models. Biometrika 95 773-778. · Zbl 1437.62527 · doi:10.1093/biomet/asn023
[18] Lin, X. (1997). Variance component testing in generalised linear models with random effects. Biometrika 84 309-326. · Zbl 0881.62074 · doi:10.1093/biomet/84.2.309
[19] Liu, Y. and Wu, Y. (2007). Variable selection via a combination of the \(L_0\) and \(L_1\) penalties. J. Comput. Graph. Statist. 16 782-798.
[20] Longford, N. T. (1993). Random Coefficient Models. Oxford Statistical Science Series 11 . Oxford Univ. Press, New York. · Zbl 0859.62064
[21] Lv, J. and Fan, Y. (2009). A unified approach to model selection and sparse recovery using regularized least squares. Ann. Statist. 37 3498-3528. · Zbl 1369.62156 · doi:10.1214/09-AOS683
[22] Patterson, H. D. and Thompson, R. (1971). Recovery of inter-block information when block sizes are unequal. Biometrika 58 545-554. · Zbl 0228.62046 · doi:10.1093/biomet/58.3.545
[23] Pu, W. and Niu, X.-F. (2006). Selecting mixed-effects models based on a generalized information criterion. J. Multivariate Anal. 97 733-758. · Zbl 1085.62083 · doi:10.1016/j.jmva.2005.05.009
[24] Qu, A. and Li, R. (2006). Quadratic inference functions for varying-coefficient models with longitudinal data. Biometrics 62 379-391. · Zbl 1097.62037 · doi:10.1111/j.1541-0420.2005.00490.x
[25] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58 267-288. · Zbl 0850.62538
[26] Vaida, F. and Blanchard, S. (2005). Conditional Akaike information for mixed-effects models. Biometrika 92 351-370. · Zbl 1094.62077 · doi:10.1093/biomet/92.2.351
[27] Verbeke, G. and Molenberghs, G. (2000). Linear Mixed Models for Longitudinal Data . Springer, New York. · Zbl 0956.62055
[28] Vrahatis, M. N. (1989). A short proof and a generalization of Miranda’s existence theorem. Proc. Amer. Math. Soc. 107 701-703. · Zbl 0695.55001 · doi:10.2307/2048168
[29] Wang, H., Li, R. and Tsai, C.-L. (2007). Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika 94 553-568. · Zbl 1135.62058 · doi:10.1093/biomet/asm053
[30] Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 49-67. · Zbl 1141.62030 · doi:10.1111/j.1467-9868.2005.00532.x
[31] Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. Ann. Statist. 38 894-942. · Zbl 1183.62120 · doi:10.1214/09-AOS729
[32] Zhang, Y., Li, R. and Tsai, C.-L. (2010). Regularization parameter selections via generalized information criterion. J. Amer. Statist. Assoc. 105 312-323. · Zbl 1397.62262 · doi:10.1198/jasa.2009.tm08013
[33] Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418-1429. · Zbl 1171.62326 · doi:10.1198/016214506000000735
[34] Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 301-320. · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x
[35] Zou, H. and Li, R. (2008). One-step sparse estimates in nonconcave penalized likelihood models (with discussion). Ann. Statist. 36 1509-1566. · Zbl 1282.62112 · doi:10.1214/009053607000000802
[36] Zou, H. and Zhang, H. H. (2009). On the adaptive elastic-net with a diverging number of parameters. Ann. Statist. 37 1733-1751. · Zbl 1168.62064 · doi:10.1214/08-AOS625
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.