×

Model averaging for linear mixed models via augmented Lagrangian. (English) Zbl 07464458

Summary: Model selection for linear mixed models has been a focus of recent research in statistics. Yet, the method of model averaging has been sparsely explored in this context. A weight finding criterion for model averaging of linear mixed models is introduced, as well as its implementation for the programming language R. Since the optimization of the underlying criterion is non-trivial, a fast and robust implementation of the augmented Lagrangian optimization technique is employed. Furthermore, the influence of the weight finding criterion on the resulting model averaging estimator is illustrated through simulation studies and two applications based on real data.

MSC:

62-XX Statistics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Akaike, H., Information Theory and an Extension of the Maximum Likelihood Principle, Proceedings of the 2nd International Symposium on Information (1973), Czaki, Akademiai Kiado: Czaki, Akademiai Kiado Budapest · Zbl 0283.62006
[2] Avriel, M., Nonlinear Programming: Analysis and Methods (2003), Courier Corporation · Zbl 1140.90002
[3] Bartolucci, F.; Bacci, S.; Pigini, C., Misspecification test for random effects in generalized linear finite-mixture models for clustered binary and ordered data, Econom. Stat., 3, 112-131 (2017)
[4] Bates, D.; Alday, P.; Kleinschmidt, D.; Calderón, J. B.S.; Noack, A.; Kelman, T.; Bouchet-Valat, M.; Gagnon, Y. L.; Babayan, S.; Mogensen, P. K.; Piibeleht, M.; Hatherly, M.; Saba, E.; Baldassari, A. (2020), Juliastats/mixedmodels.jl, v2.3.0
[5] Bates, D.; Mächler, M.; Bolker, B.; Walker, S., Fitting linear mixed-effects models using lme4, J. Stat. Softw., 67, 1-48 (2014)
[6] Buckland, S. T.; Burnham, K. P.; Augustin, N. H., Model selection: an integral part of inference, Biometrics, 603-618 (1997) · Zbl 0885.62118
[7] Cantoni, E.; Jacot, N.; Ghisletta, P., Review and comparison of measures of explained variation and model selection in linear mixed-effects models, Econom. Stat. (2021)
[8] Crainiceanu, C. M.; Ruppert, D., Likelihood ratio tests in linear mixed models with one variance component, J. R. Stat. Soc., Ser. B, Stat. Methodol., 66, 165-185 (2004) · Zbl 1061.62027
[9] Di, C. Z.; Crainiceanu, C. M.; Caffo, B. S.; Punjabi, N. M., Multilevel functional principal component analysis, Ann. Appl. Stat., 3, 458 (2009) · Zbl 1160.62061
[10] Efron, B., The estimation of prediction error: covariance penalties and cross-validation, J. Am. Stat. Assoc., 99, 619-632 (2004) · Zbl 1117.62324
[11] Eilers, P. H.C.; Marx, B. D., Flexible smoothing with b -splines and penalties, Stat. Sci., 11, 89-121 (1996) · Zbl 0955.62562
[12] Fahrmeir, L.; Kneib, T.; Lang, S.; Marx, B., Regression (2013), Springer Berlin Heidelberg: Springer Berlin Heidelberg Berlin, Heidelberg · Zbl 1276.62046
[13] Ghalanos, A., Theussl, S., 2012. Package ‘rsolnp’.
[14] Ghalanos, A., Theussl, S., 2015. Rsolnp: General Non-linear Optimization Using Augmented Lagrange Multiplier Method. R package version 1.16.
[15] Greven, S.; Kneib, T., On the behaviour of marginal and conditional aic in linear mixed models, Biometrika, 97, 773-789 (2010) · Zbl 1204.62114
[16] Griesbach, C.; Säfken, B.; Waldmann, E., Gradient boosting for linear mixed models, Int. J. Biostat., 20200136 (2021)
[17] Guo, W., Functional mixed effects models, Biometrics, 58, 121-128 (2002) · Zbl 1209.62072
[18] Harville, D., Extension of the gauss-markov theorem to include the estimation of random effects, Ann. Stat., 4, 384-395 (1976) · Zbl 0323.62043
[19] Henderson, C. R., Estimation of genetic parameters, (Biometrics, International Biometric Soc 1441 I ST, NW, Suite 700, Washington, DC 20005-2210 (1950)), 186-187
[20] Hestenes, M. R., Multiplier and gradient methods, J. Optim. Theory Appl., 4, 303-320 (1969) · Zbl 0174.20705
[21] Ho, H. J.; Lin, T. I., Robust linear mixed models using the skew t distribution with application to schizophrenia data, Biom. J., 52, 449-469 (2010) · Zbl 1197.62055
[22] Kneib, T.; Silbersdorff, A.; Säfken, B., Rage against the mean – a review of distributional regression approaches, Econom. Stat. (2021)
[23] Kullback, S.; Leibler, R. A., On information and sufficiency, Ann. Math. Stat., 22, 79-86 (1951) · Zbl 0042.38403
[24] Kuznetsova, A.; Brockhoff, P. B.; Christensen, R. H.B., lmerTest package: tests in linear mixed effects models, J. Stat. Softw., 82 (2017)
[25] Laird, N. M.; Ware, J. H., Random-effects models for longitudinal data, Biometrics, 963-974 (1982) · Zbl 0512.62107
[26] Li, C.; Yin, W.; Jiang, H.; Zhang, Y., An efficient augmented lagrangian method with applications to total variation minimization, Comput. Optim. Appl., 56, 507-530 (2013) · Zbl 1287.90066
[27] Liang, H.; Wu, H.; Zou, G., A note on conditional aic for linear mixed-effects models, Biometrika, 95, 773-778 (2008) · Zbl 1437.62527
[28] Lin, T. I.; Lee, J. C., A robust approach to t linear mixed models applied to multiple sclerosis data, Stat. Med., 25, 1397-1412 (2006)
[29] Luenberger, D. G.; Ye, Y., Linear and Nonlinear Programming, Vol. 2 (2008), Springer · Zbl 1207.90003
[30] Nocedal, J.; Wright, S., Numerical Optimization (2006), Springer Science & Business Media · Zbl 1104.65059
[31] Potthoff, R. F.; Roy, S., A generalized multivariate analysis of variance model useful especially for growth curve problems, Biometrika, 51, 313-326 (1964) · Zbl 0138.14306
[32] Powell, M. J., A method for nonlinear constraints in minimization problems, Optimization, 283-298 (1969) · Zbl 0194.47701
[33] R: A Language and Environment for Statistical Computing (2019), R Foundation for Statistical Computing: R Foundation for Statistical Computing Vienna, Austria
[34] Ruppert, D.; Wand, M. P.; Carroll, R. J., Semiparametric Regression. 12 (2003), Cambridge University Press · Zbl 1038.62042
[35] Ruszczynski, A., Nonlinear Optimization (2011), Princeton University Press
[36] Saefken, B.; Kneib, T.; van Waveren, C. S.; Greven, S., A unifying approach to the estimation of the conditional akaike information in generalized linear mixed models, Electron. J. Stat., 8, 201-225 (2014) · Zbl 1282.62168
[37] Säfken, B.; Rügamer, D.; Kneib, T.; Greven, S., Conditional model selection in mixed-effects models with caic4 (2018)
[38] Schwarz, G., Estimating the dimension of a model, Ann. Stat., 6, 461-464 (1978) · Zbl 0379.62005
[39] Self, S. G.; Liang, K. Y., Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions, J. Am. Stat. Assoc., 82, 605-610 (1987) · Zbl 0639.62020
[40] Silbersdorff, A.; Lynch, J.; Klasen, S.; Kneib, T., Reconsidering the income-health relationship using distributional regression, Health Econ., 27, 1074-1088 (2018)
[41] Stein, C., A bound for the error in the normal approximation to the distribution of a sum of dependent random variables, (Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Volume 2: Probability Theory (1972), The Regents of the University of California) · Zbl 0278.60026
[42] Säfken, B.; Kneib, T., Conditional covariance penalties for mixed models, Scand. J. Stat., 47, 990-1010 (2020) · Zbl 1476.62152
[43] Vaida, F.; Blanchard, S., Conditional akaike information for mixed-effects models, Biometrika, 92, 351-370 (2005) · Zbl 1094.62077
[44] Varadhan, R., Alabama: constrained nonlinear optimization (2015), r package version 2015.3-1
[45] Verbeke, G.; Molenberghs, G., Linear Mixed Models for Longitudinal Data (2009), Springer Science & Business Media · Zbl 1162.62070
[46] Wager, C.; Vaida, F.; Kauermann, G., Model selection for penalized spline smoothing using akaike information criteria, Aust. N. Z. J. Stat., 49, 173-190 (2007) · Zbl 1117.62073
[47] Wood, S. N., A simple test for random effects in regression models, Biometrika, 100, 1005-1010 (2013) · Zbl 1279.62149
[48] Wood, S.N., 2017. Generalized additive models: an introduction with R. CRC press. · Zbl 1368.62004
[49] Wood, S. N.; Pya, N.; Säfken, B., Smoothing parameter and model selection for general smooth models, J. Am. Stat. Assoc., 111, 1548-1563 (2016)
[50] Ye, J., On measuring and correcting the effects of data mining and model selection, J. Am. Stat. Assoc., 93, 120-131 (1998) · Zbl 0920.62056
[51] Zhang, X.; Zou, G.; Liang, H., Model averaging and weight choice in linear mixed-effects models, Biometrika, 101, 205-218 (2014) · Zbl 1285.62077
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.