Maximum likelihood degree of variance component models.

*(English)*Zbl 1281.62159Summary: Most statistical software packages implement numerical strategies for computation of maximum likelihood estimates in random effects models. Little is known, however, about the algebraic complexity of this problem. For the one-way layout with random effects and unbalanced group sizes, we give formulas for the algebraic degree of the likelihood equations as well as the equations for restricted maximum likelihood estimation. In particular, the latter approach is shown to be algebraically less complex. The formulas are obtained by studying a univariate rational equation whose solutions correspond to the solutions of the likelihood equations. Applying techniques from computational algebra, we also show that balanced two-way layouts with or without interaction have likelihood equations of degree four. Our work suggests that algebraic methods allow one to reliably find global optima of likelihood functions of linear mixed models with a small number of variance components.

##### Keywords:

analysis of variance; linear mixed model; maximum likelihood; restricted maximum likelihood; variance component##### Software:

faraway##### References:

[1] | Quentin D. Atkinson, Phonemic diversity supports a serial founder effect model of language expansion from Africa , Science 332 (2011), no. 6027, 346-349. |

[2] | Max-Louis G. Buot, Serkan Hoşten, and Donald St. P. Richards, Counting and locating the solutions of polynomial systems of maximum likelihood equations. II. The Behrens-Fisher problem , Statist. Sinica 17 (2007), no. 4, 1343-1354. · Zbl 1132.62041 |

[3] | Fabrizio Catanese, Serkan Hoşten, Amit Khetan, and Bernd Sturmfels, The maximum likelihood degree , Amer. J. Math. 128 (2006), no. 3, 671-697. · Zbl 1123.13019 |

[4] | Owen L. Davies and Peter L. Goldsmith (eds.), Statistical methods in research and production , 4th ed., Hafner, 1972. |

[5] | Eugene Demidenko and Hélène Massam, On the existence of the maximum likelihood estimate in variance components models , Sankhyā Ser. A 61 (1999), no. 3, 431-443. · Zbl 1242.62058 |

[6] | Mathias Drton, Bernd Sturmfels, and Seth Sullivant, Lectures on algebraic statistics , Birkhäuser Verlag AG, Basel, Switzerland, 2009. · Zbl 1166.13001 |

[7] | Julian J. Faraway, Extending the linear model with R , Texts in Statistical Science Series, Chapman & Hall/CRC, Boca Raton, FL, 2006, Generalized linear, mixed effects and nonparametric regression models. · Zbl 1095.62082 |

[8] | Serkan Hoşten, Amit Khetan, and Bernd Sturmfels, Solving the likelihood equations , Found. Comput. Math. 5 (2005), no. 4, 389-407. · Zbl 1097.13035 |

[9] | R. R. Hocking, The analysis of linear models , Brooks/Cole Publishing Co., Monterey, CA, 1985. · Zbl 0625.62054 |

[10] | Serkan Hoşten and Seth Sullivant, The algebraic complexity of maximum likelihood estimation for bivariate missing data , Algebraic and geometric methods in statistics, Cambridge Univ. Press, Cambridge, 2010, pp. 123-133. |

[11] | Jiming Jiang, Linear and generalized linear mixed models and their applications , Springer Series in Statistics, Springer, New York, 2007. · Zbl 1152.62040 |

[12] | P. McCullagh and J. A. Nelder, Generalized linear models , 2nd ed., Monographs on Statistics and Applied Probability, Chapman & Hall, London, 1989. · Zbl 0744.62098 |

[13] | Shayle R. Searle, George Casella, and Charles E. McCulloch, Variance components , Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics, John Wiley & Sons Inc., New York, 1992, A Wiley-Interscience Publication. · Zbl 1108.62064 |

[14] | William H. Swallow and John F. Monahan, Monte Carlo comparison of ANOVA, MIVQUE, REML, and ML estimators of variance components , Technometrics 26 (1984), no. 1, 47-57. · Zbl 0548.62051 |

[15] | Hardeo Sahai and Mario Miguel Ojeda, Analysis of variance for random models. Vol. I. Balanced data , Birkhäuser Boston Inc., Boston, MA, 2004, Theory, methods, applications and data analysis. · Zbl 1076.62075 |

[16] | Hardeo Sahai and Mario Miguel Ojeda, Analysis of variance for random models. Vol. II. Unbalanced data , Birkhäuser Boston Inc., Boston, MA, 2005, Theory, methods, applications, and data analysis. · Zbl 1077.62055 |

[17] | Bernd Sturmfels, Open problems in algebraic statistics , Emerging applications of algebraic geometry, IMA Vol. Math. Appl., vol. 149, Springer, New York, 2009, pp. 351-363. · Zbl 1158.13300 |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.