Choosing initial values for the EM algorithm for finite mixtures. (English) Zbl 1429.62082

Summary: The EM algorithm is the standard tool for maximum likelihood estimation in finite mixture models. The main drawbacks of the EM algorithm are its slow convergence and the dependence of the solution on both the stopping criterion and the initial values used. The problems referring to slow convergence and the choice of a stopping criterion have been dealt with in literature and the present paper deals with the initial value problem for the EM algorithm. The aim of this paper is to compare several methods for choosing initial values for the EM algorithm in the case of finite mixtures as well as to propose some new methods based on modifications of existing ones. The cases of finite normal mixtures with common variance and finite Poisson mixtures are examined through a simulation study.


62F10 Point estimation
Full Text: DOI


[1] Atwood, L.D; Wilson, A.F; Elston, R.C; Bailey-Wilson, J.E, Computational aspects of Fitting a mixture of two normal distributions using maximum likelihood, Comm. statist. simulation comput., 21, 769-781, (1992)
[2] Böhning, D, Computer assisted analysis of mixtures & applications in meta-analysis, disease mapping & others, (1999), CRC Press Boca Raton, FL
[3] Böhning, D; Dietz, E; Schaub, R; Schlattman, P; Lindsay, B, The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family, Ann. inst. statist. math., 46, 373-388, (1994) · Zbl 0802.62017
[4] Dempster, A.P; Laird, N.M; Rubin, D, Maximum likelihood from incomplete data via the EM algorithm, J. roy. statist. soc. ser. B, 39, 1-38, (1977) · Zbl 0364.62022
[5] Finch, S; Mendell, N; Thode, H, Probabilistic measures of adequacy of a numerical search for a global maximum, J. amer. statist. assoc., 84, 1020-1023, (1989)
[6] Fowlkes, E, Some methods for studying the mixture of two normal (lognormal) distributions, J. amer. statist. assoc., 74, 561-575, (1979) · Zbl 0434.62024
[7] Furman, W.D; Lindsay, B.G, Testing for the number of components in a mixture of normal distributions using moment estimators, Comput. statist. data. anal., 17, 473-492, (1994) · Zbl 0937.62557
[8] Furman, W.D; Lindsay, B.G, Measuring the relative effectiveness of moment estimators as starting values in maximizing likelihoods, Comput. statist. and data. anal., 17, 493-508, (1994) · Zbl 0937.62532
[9] Karlis, D; Xekalaki, E, On testing for the number of components in finite Poisson mixtures, Ann. inst. statist. math., 51, 149-162, (1999) · Zbl 0928.62061
[10] Karlis, D., Xekalaki, E., 2001. On implementing the EM algorithm in finite Poisson mixtures. Department of Statistics, Athens University of Economics, p. 132. · Zbl 0997.62042
[11] Laird, N, Nonparametric maximum likelihood estimation of a mixing distribution, J. amer. statist. assoc., 73, 805-811, (1978) · Zbl 0391.62029
[12] Leroux, B.G, Consistent estimation of a mixing distribution, Ann. statist., 20, 1350-1360, (1992) · Zbl 0763.62015
[13] Lindsay, B.G, The geometry of mixture likelihood. a general theory, Ann. statist., 11, 86-94, (1983) · Zbl 0512.62005
[14] Lindsay, B.G, Mixture models: theory, geometry and applications, NSF-CBMS regional conference series in probability and statistics, Vol. 5, (1995), Institute of Mathematical Statistics California · Zbl 1163.62326
[15] Lindsay, B.G; Basak, P, Multivariate normal mixturesa fast consistent method of moments, J. amer. statist. assoc., 88, 468-475, (1993) · Zbl 0773.62037
[16] Lindstrom, M.J; Bates, D.M, Newton-raphson and EM algorithms for linear mixed effects models for repeated measures data, J. amer. statist. assoc., 83, 1014-1022, (1988) · Zbl 0671.65119
[17] Markatou, M, Mixture models, robustness and the weighted likelihood methodology, Biometrics, 56, 483-486, (2000) · Zbl 1060.62511
[18] Markatou, M; Basu, A; Lindsay, B.G, Weighted likelihood estimating equations with a bootstrap root search, J. amer. statist. assoc., 93, 740-750, (1998) · Zbl 0918.62046
[19] McLachlan, G.J, On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture, Appl. statist., 36, 318-324, (1987)
[20] McLachlan, G.J, On the choice of initial values for the EM algorithm in Fitting mixture models, The Statistician, 37, 417-425, (1988)
[21] McLachlan, G.J; Peel, D, Finite mixture models, (2000), Wiley New York · Zbl 0963.62061
[22] Pilla, R.S; Lindsay, B.G, Alternative EM methods for nonparametric finite mixture models, Biometrika, 88, 535-550, (2001) · Zbl 0984.62024
[23] Pilla, R.S; Kamarthi, S.V; Lindsay, B.G, Aitken-based acceleration methods for assessing convergence of multilayer neural networks, IEEE trans. neur. net., 12, 998-1012, (2001)
[24] Seidel, W; Mosler, K; Alker, M, A cautionary note on likelihood ratio tests in mixture models, Ann. inst. statist. math., 52, 481-487, (2000) · Zbl 0960.62025
[25] Seidel, W; Mosler, K; Alker, M, Likelihood ratio tests based on subglobal optimisationa power comparison in exponential mixture models, Statist. hefte, 41, 85-98, (2000) · Zbl 0941.62021
[26] Seidel, W., Sevcikova, H., Alker, M., 2000c. On the power of different versions of the likelihood ratio test for homogeneity in an exponential mixture model. Dept. Stat. und Quantit. Ökon. Universität der Bundeswehr Hamburg, Report 92.
[27] Woodward, W; Parr, W; Schucany, R; Lindsey, H, A comparison of minimum distance and maximum likelihood estimation of a mixture proportion, J. amer. statist. assoc., 79, 590-598, (1984) · Zbl 0547.62017
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.