##
**Linear mixed models and penalized least squares.**
*(English)*
Zbl 1051.62063

Summary: Linear mixed-effects models are an important class of statistical models that are used directly in many fields of applications and also are used as iterative steps in fitting other types of mixed-effects models, such as generalized linear mixed models. The parameters in these models are typically estimated by maximum likelihood or restricted maximum likelihood. In general, there is no closed-form solution for these estimates and they must be determined by iterative algorithms such as EM iterations or general nonlinear optimization.

Many of the intermediate calculations for such iterations have been expressed as generalized least squares problems. We show that an alternative representation as a penalized least squares problem has many advantages, the computational properties including the ability to evaluate explicitly a profiled log-likelihood or log-restricted likelihood, the gradient and Hessian of this profiled objective, and an ECME update to refine this objective.

Many of the intermediate calculations for such iterations have been expressed as generalized least squares problems. We show that an alternative representation as a penalized least squares problem has many advantages, the computational properties including the ability to evaluate explicitly a profiled log-likelihood or log-restricted likelihood, the gradient and Hessian of this profiled objective, and an ECME update to refine this objective.

### MSC:

62J10 | Analysis of variance and covariance (ANOVA) |

62H12 | Estimation in multivariate analysis |

62J12 | Generalized linear models (logistic models) |

65C60 | Computational problems in statistics (MSC2010) |

### Keywords:

REML; Gradient; Hessian; EM algorithm; ECME algorithm; Maximum likelihood; Profile likelihood; Multilevel models### Software:

R
PDFBibTeX
XMLCite

\textit{D. M. Bates} and \textit{S. DebRoy}, J. Multivariate Anal. 91, No. 1, 1--17 (2004; Zbl 1051.62063)

Full Text:
DOI

### References:

[1] | S. DebRoy, Computational methods for mixed-effects models, Ph.D. Thesis, University of Wisconsin-Madison, 2003.; S. DebRoy, Computational methods for mixed-effects models, Ph.D. Thesis, University of Wisconsin-Madison, 2003. |

[2] | Dempster, A. P.; Laird, N. M.; Rubin, D. B., Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc. Ser. B, 39, 1-22 (1977) · Zbl 0364.62022 |

[3] | Golub, G. H.; Pereyra, V., The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate, SIAM J. Numer. Anal., 10, 2, 413-432 (1973) · Zbl 0258.65045 |

[4] | C. Henderson, Applications of Linear Models in Animal Breeding, University of Guelph, Ontario, Canada, 1984.; C. Henderson, Applications of Linear Models in Animal Breeding, University of Guelph, Ontario, Canada, 1984. |

[5] | Ihaka, R.; Gentleman, R., Ra language for data analysis and graphics, J. Comput. Graphical Statist., 5, 3, 299-314 (1996) |

[6] | Liu, C.; Rubin, D. B., The ECME algorithma simple extension of EM and ECM with faster monotone convergence, Biometrika, 81, 633-648 (1994) · Zbl 0812.62028 |

[7] | Pinherio, J. C.; Bates, D. M., Mixed-effects models in \(S\) and \(S−PLUS\), Statistics and Computing (2000), Springer: Springer Berlin · Zbl 0953.62065 |

[8] | Raudenbush, S. W.; Bryk, A. S., Hierarchical Linear Models: Applications and Data Analysis Methods (2002), Sage: Sage Beverley Hills, CA · Zbl 1001.62004 |

[9] | Wright, S. J., Primal-dual Interior Point Methods (1997), SIAM: SIAM Philadelphia · Zbl 0863.65031 |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.