Adaptive fitting of linear mixed-effects models with correlated random effects. (English) Zbl 1453.62575

Summary: Linear mixed-effects model has been widely used in longitudinal data analyses. In practice, the fitting algorithm can fail to converge due to boundary issues of the estimated random-effects covariance matrix \(G\), that is, being near-singular, non-positive definite, or both. Current available algorithms are not computationally optimal because the condition number of matrix \(G\) is unnecessarily increased when the random-effects correlation estimate is not zero. We propose an adaptive fitting (AF) algorithm using an optimal linear transformation of the random-effects design matrix. It is a data-driven adaptive procedure, aiming at reducing subsequent random-effects correlation estimates down to zero in the optimal transformed estimation space. Simulations show that AF significantly improves the convergent properties, especially under small sample size, relative large noise and high correlation settings. One real data for insulin-like growth factor protein is used to illustrate the application of this algorithm implemented with software package R (nlme).


62J05 Linear regression; mixed models
62J10 Analysis of variance and covariance (ANOVA)
62-08 Computational methods for problems pertaining to statistics


MEMSS; R; nlme; S-PLUS; WWGbook
Full Text: DOI Link


[1] N.M. Laird and J.H. Ware, Random-effects models for longitudinal data, Biometrics 38 (1982), pp. 963-974. doi: 10.2307/2529876[Crossref], [PubMed], [Web of Science ®], [Google Scholar] · Zbl 0512.62107
[2] H. Goldstein, Multilevel mixed linear model analysis using iterative generalized least squares, Biometrika 73 (1986), pp. 43-56. doi: 10.1093/biomet/73.1.43[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0587.62143
[3] N.T. Longford, Random Coefficient Models, Clarendon Press, Oxford, 1993. [Google Scholar] · Zbl 0859.62064
[4] S.W. Raudenbush and A.S. Bryk, Hierarchical Linear Models: Applications and Data Analysis Methods, 2nd ed., Sage Publications, Thousand Oaks, CA, 2002. [Google Scholar]
[5] A.P. Dempster, M.R. Selwyn, C.M. Patel, and A.J. Roth, Statistical and computational aspects of mixed model analysis, Appl. Statist. 33 (1984), pp. 203-214. doi: 10.2307/2347446[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0568.62067
[6] N. Laird, N. Lange, and D. Stram, Maximum-likelihood computations with repeated measures – application of the EM-algorithm, J. Am. Statist. Assoc. 82 (1987), pp. 97-105. doi: 10.1080/01621459.1987.10478395[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 0613.62063
[7] C. Liu and D.B. Rubin, The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence, Biometrika 81 (1994), pp. 633-648. doi: 10.1093/biomet/81.4.633[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0812.62028
[8] X.L. Meng and D. van Dyk, Fast EM-type implementations for mixed effects models, J. R. Statist. Soc. Ser. B Statist. Methodol. 60 (1998), pp. 559-578. doi: 10.1111/1467-9868.00140[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0909.62073
[9] R. Jennrich and M. Schluchter, Unbalanced repeated-measures models with structured covariance matrices, Biometrics 42 (1986), pp. 805-820. doi: 10.2307/2530695[Crossref], [PubMed], [Web of Science ®], [Google Scholar] · Zbl 0625.62052
[10] R. Thompson and K. Meyer, Estimation of variance components: What is missing in the EM algorithm, J. Statist. Comput. Simul. 24 (1986), pp. 215-230. doi: 10.1080/00949658608810905[Taylor & Francis Online], [Google Scholar]
[11] M.J. Lindstrom and D.M. Bates, Newton-Raphson and EM algorithms for linear mixed-effects models for repeated-measures data, J. Am. Statist. Assoc. 83 (1988), pp. 1014-1022. [Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 0671.65119
[12] T.P. Callanan and D.A. Harville, Some new algorithms for computing restricted maximum likelihood estimates of variance components, J. Statist. Comput. Simul. 38 (1991), pp. 239-259. doi: 10.1080/00949659108811332[Taylor & Francis Online], [Google Scholar] · Zbl 0800.62423
[13] N.T. Longford, A fast scoring algorithm for maximum-likelihood-estimation in unbalanced mixed models with nested random effects, Biometrika 74 (1987), pp. 817-827. doi: 10.1093/biomet/74.4.817[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0628.62075
[14] B.T. West, K.B. Welch, and A.T. Galecki, Linear Mixed Models: A Practical Guide Using Statistical Software, Chapman and Hall/CRC, Boca Raton, FL, 2006. [Crossref], [Google Scholar] · Zbl 1269.62057
[15] D.A. van Dyk, Fitting mixed-effects models using efficient EM-type algorithms, J. Comput. Graph. Stat. 9 (2000), pp. 78-98. [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
[16] W.J. Browne and D. Draper, Implementation and performance issues in the Bayesian and likelihood fitting of multilevel models, Comput. Stat. 15 (2000), pp. 391-420. doi: 10.1007/s001800000041[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1037.62013
[17] J. Berkhof and T. Snijders, Variance component testing in multilevel models, J. Educ. Behav. Stat. 26 (2001), pp. 133-152. doi: 10.3102/10769986026002133[Crossref], [Web of Science ®], [Google Scholar]
[18] Y. Shieh and R. Fouladi, The effect of multicollinearity on multilevel modeling parameter estimates and standard errors, Educ. Psychol. Meas. 63 (2003), pp. 951-985. doi: 10.1177/0013164403258402[Crossref], [Web of Science ®], [Google Scholar]
[19] S.K. Mikulich, G.O. Zerbe, R.H. Jones, and T.J. Crowley, Relating the classical covariance adjustment techniques of multivariate growth curve models to modern univariate mixed effects models, Biometrics 55 (1999), pp. 957-964. doi: 10.1111/j.0006-341X.1999.00957.x[Crossref], [PubMed], [Web of Science ®], [Google Scholar] · Zbl 1059.62678
[20] A. Pryseley, C. Tchonlafi, G. Verbeke, and G. Molenberghs, Estimating negative variance components from Gaussian and non-Gaussian data: A mixed models approach, Comput. Stat. Data Anal. 55 (2011), pp. 1071-1085. doi: 10.1016/j.csda.2010.09.002[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1284.62060
[21] J.C. Pinheiro and D.M. Bates, Mixed-Effects Models in S and S-PLUS, Springer, New York, 2000. [Crossref], [Google Scholar] · Zbl 0953.62065
[22] L. Gurrin, K. Blake, S. Evans, and J. Newnham, Statistical measures of foetal growth using linear mixed models applied to the foetal origins hypothesis, Stat. Med. 20 (2001), pp. 3391-3409. doi: 10.1002/sim.891[Crossref], [PubMed], [Web of Science ®], [Google Scholar]
[23] N. Solaro and P.A. Ferrari, Robustness of parameter estimation procedures in multilevel models when random effects are MEP distributed, Statist. Methods Appl. 16 (2007), pp. 51-67. doi: 10.1007/s10260-006-0016-6[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1157.62034
[24] I.G.G. Kreft, J. Leeuwde, and L. Aiken, The effect of different forms of centering in hierarchical linear modeling, Multivariate Behav. Res. 30 (1995), pp. 1-20. doi: 10.1207/s15327906mbr3001_1[Taylor & Francis Online], [Web of Science ®], [Google Scholar]
[25] R. van der Leeden, K. Vrijburg, and J. Leeuwde, A review of two different approaches for the analysis of growth data using longitudinal mixed linear models, Comput. Stat. Data Anal. 21 (1996), pp. 583-605. doi: 10.1016/0167-9473(96)82296-1[Crossref], [Web of Science ®], [Google Scholar]
[26] C.H. Morrell, J.D. Pearson, and L.J. Brant, Linear transformations of linear mixed-effects models, Am. Statist. 51 (1997), pp. 338-343. [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
[27] D. Zhang and M. Davidian, Linear mixed models with flexible distributions of random effects for longitudinal data, Biometrics 57 (2001), pp. 795-802. doi: 10.1111/j.0006-341X.2001.00795.x[Crossref], [PubMed], [Web of Science ®], [Google Scholar] · Zbl 1209.62087
[28] D.W. Marquardt and R.D. Snee, Ridge regression in practice, Am. Statist. 29 (1975), pp. 3-20. [Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 0361.62060
[29] R.A. Bradley and S.S. Srivastava, Correlation in polynomial regression, Am. Statist. 33 (1979), pp. 11-14. [Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 0392.62050
[30] N.R. Draper and H. Smith, Applied Regression Analysis, 3rd ed., Wiley-Interscience, New York, NY, 1998. [Crossref], [Google Scholar] · Zbl 0895.62073
[31] H. Jacqmin-Gadda, S. Sibillot, C. Proust, J.M. Molina, and R. Thiebaut, Robustness of the linear mixed model to misspecified error distribution, Comput. Stat. Data Anal. 51 (2007), pp. 5142-5154. doi: 10.1016/j.csda.2006.05.021[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1162.62319
[32] H. Goldstein, Multilevel Statistical Models, 3rd ed., Wiley, New York, NY, 2002. [Google Scholar] · Zbl 1014.62126
[33] Y.T. Hwang and P.F. Wei, A novel method for testing normality in a mixed model of a nested classification, Comput. Stat. Data Anal. 51 (2006), pp. 1163-1183. doi: 10.1016/j.csda.2005.11.014[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1157.62397
[34] D.A. Belsley and R.W. Oldford, The general problem of ill conditioning and its role in statistical analysis, Comput. Stat. Data Anal. 4 (1986), pp. 103-120. doi: 10.1016/0167-9473(86)90014-9[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0608.62087
[35] D. Sengupta and P. Bhimasankaram, On the roles of observations in collinearity in the linear model, J. Am. Statist. Assoc. 92 (1997), pp. 1024-1032. doi: 10.1080/01621459.1997.10474058[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 0888.62075
[36] L.N. Trefethen and D. Bau, Numerical Linear Algebra, Society for Industrial and Applied Mathematics, SIAM, Philadelphia, PA, 1997. [Crossref], [Google Scholar] · Zbl 0874.65013
[37] K.H. Yuan and W. Chan, Structural equation modeling with near singular covariance matrices, Comput. Stat. Data Anal. 52 (2008), pp. 4842-4858. doi: 10.1016/j.csda.2008.03.030[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1452.62427
[38] W.W. Chen, C.M. Hurvich, and Y. Lu, On the correlation matrix of the discrete Fourier transform and the fast solution of large Toeplitz systems for long-memory time series, J. Am. Statist. Assoc. 101 (2006), pp. 812-822. doi: 10.1198/016214505000001069[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 1119.62358
[39] J.C. Pinheiro, D.M. Bates, S. DebRoy, D. Sarkar, and the R Development Core Team, nlme: Linear and Nonlinear Mixed Effects Models, R package version 2.9.2, 2009. [Google Scholar]
[40] M. Davidian and D.M. Giltinan, Nonlinear Models for Repeated Measurement Data, Chapman and Hall, London, 1995. [Google Scholar]
[41] J.C. Pinheiro and D.M. Bates, Unconstrained parametrizations for variance-covariance matrices, Stat. Comput. 6 (1996), pp. 289-296. doi: 10.1007/BF00140873[Crossref], [Web of Science ®], [Google Scholar]
[42] R. Abellana, J. Carrasco, L. Jover, and C. Ascaso, Improving the convergence rate in conditional autoregressive models, Comput. Stat. Data Anal. 50 (2006), pp. 1153-1163. doi: 10.1016/j.csda.2004.11.014[Crossref], [Web of Science ®], [Google Scholar] · Zbl 1431.62488
[43] G. Verbeke and E. Lesaffre, The effect of misspecifying the random-effects distribution in linear mixed models for longitudinal data, Comput. Stat. Data Anal. 23 (1997), pp. 541-556. doi: 10.1016/S0167-9473(96)00047-3[Crossref], [Web of Science ®], [Google Scholar] · Zbl 0900.62374
[44] G. Verbeke, E. Lesaffre, and L.J. Brant, The detection of residual serial correlation in linear mixed models, Stat. Med. 17 (1998), pp. 1391-1402. doi: 10.1002/(SICI)1097-0258(19980630)17:12<1391::AID-SIM851>3.0.CO;2-4[Crossref], [PubMed], [Web of Science ®], [Google Scholar]
[45] N. Lange and N.M. Laird, The effect of covariance structure on variance-estimation in balanced growth-curve models with random parameters, J. Am. Statist. Assoc. 84 (1989), pp. 241-247. doi: 10.1080/01621459.1989.10478761[Taylor & Francis Online], [Web of Science ®], [Google Scholar] · Zbl 0678.62070
[46] H. Schielzeth and W. Forstmeier, Conclusions beyond support: Overconfident estimates in mixed models, Behav. Ecol. 20 (2009), pp. 416-420. doi: 10.1093/beheco/arn145[Crossref], [PubMed], [Web of Science ®], [Google Scholar]
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.