Avoiding spurious local maximizers in mixture modeling. (English) Zbl 1331.62100

Summary: The maximum likelihood estimation in the finite mixture of distributions setting is an ill-posed problem that is treatable, in practice, through the EM algorithm. However, the existence of spurious solutions (singularities and non-interesting local maximizers) makes difficult to find sensible mixture fits for non-expert practitioners. In this work, a constrained mixture fitting approach is presented with the aim of overcoming the troubles introduced by spurious solutions. Sound mathematical support is provided and, which is more relevant in practice, a feasible algorithm is also given. This algorithm allows for monitoring solutions in terms of the constant involved in the restrictions, which yields a natural way to discard spurious solutions and a valuable tool for data analysts.


62F10 Point estimation


Full Text: DOI Link


[1] Anderson, E.: The irises of the Gaspe Peninsula. Bull. Am. Iris Soc. 59, 2-5 (1935) · Zbl 0704.62103
[2] Banfield, JD; Raftery, AE, Model-based gaussian and non-Gaussian clustering, Biometrics, 49, 803-821, (1993) · Zbl 0794.62034
[3] Celeux, G., Govaert, G.: Gaussian parsimonious clustering models. Pattern Recognition 28, 781-793 (1995) · Zbl 0891.62020
[4] Celeux, G., Diebolt, J.: The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Comput. Stat. Quater. 2, 73-82 (1985)
[5] Chen, J; Tan, X, Inference for multivariate normal mixtures, J. Multivar. Anal., 100, 1367-1383, (2009) · Zbl 1162.62052
[6] Ciuperca, G; Ridolfi, A; Idier, J, Penalized maximum likelihood estimator for normal mixtures, Scand. J. Stat., 30, 45-59, (2003) · Zbl 1034.62018
[7] Coretto, P; Hennig, C, A simulations study to compare robust clustering methods based on mixtures, Adv. Data Anal. Classif., 4, 111-135, (2010) · Zbl 1284.62366
[8] Day, NE, Estimating the components of a mixture of two normal distributions, Biometrika, 56, 463-474, (1969) · Zbl 0183.48106
[9] Dempster, AP; Laird, NM; Rubin, DB, Maximum likelihood from incomplete data via the EM algorithm, J. R. Stat. Soc., 39, 1-38, (1977) · Zbl 0364.62022
[10] Dennis, JE; Powell, MJD (ed.), Algorithms for nonlinear Fitting, (1982), Cambridge · Zbl 0545.65007
[11] Dykstra, R.L.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc. 78, 837-842 (1983) · Zbl 0535.62063
[12] Fisher, RA, The use of multiple measurements in taxonomic problems, Ann. Eugen., 7, 179-188, (1936)
[13] Fraley, C; Raftery, AE, Bayesian regularization for normal mixture estimation and model-based clustering, J. Classif., 24, 155-181, (2007) · Zbl 1159.62302
[14] Fritz, H; García-Escudero, LA; Mayo-Iscar, A, A fast algorithm for robust constrained clustering, Comput. Stat. Data Anal., 61, 124-136, (2013) · Zbl 1349.62264
[15] Gallegos, MT; Ritter, G, Trimming algorithms for clustering contaminated grouped data and their robustness, Adv. Data Anal. Classif., 3, 135-167, (2009) · Zbl 1284.62372
[16] Gallegos, MT; Ritter, G, Trimmed ML estimation of contaminated mixtures, Sankhya, 71, 164-220, (2009) · Zbl 1193.62021
[17] García-Escudero, LA; Gordaliza, A; Matrán, C; Mayo-Iscar, A, A general trimming approach to robust cluster analysis, Ann. Stat., 36, 1324-1345, (2008) · Zbl 1360.62328
[18] Greselin, F; Ingrassia, S; Fink, A (ed.); Lausen, B (ed.); Seidel, W (ed.); Ultsch, A (ed.), Weakly homoscedastic constraints for mixtures of \(t\)-distributions, 219-228, (2010), Berlin
[19] Hathaway, R.J.: Constrained maximum-likelihood estimation for a mixture of \(m\) univariate normal distributions. Ph.D. Dissertation, Department of Mathematical Sciences, Rice University (1983)
[20] Hathaway, R.J.: A constrained formulation of maximum likelihood estimation for normal mixture distributions. Ann. Stat. 13, 795-800 (1985) · Zbl 0576.62039
[21] Hathaway, RJ, A constrained EM algorithm for univariate normal mixtures, J. Stat. Comput. Simul., 23, 211-230, (1986)
[22] Hennig, C, Breakdown points for maximum likelihood estimators of location-scale mixtures, Ann. Stat., 32, 1313-1340, (2004) · Zbl 1047.62063
[23] Ingrassia, S; Rocci, R, Constrained monotone EM algorithms for finite mixture of multivariate gaussians, Comput. Stat. Data Anal., 51, 5339-5351, (2007) · Zbl 1445.62116
[24] Maitra, R.: Initializing partition-optimization algorithms. IEEE/ACM Trans. Comput. Biol. Bioinf. 6, 144-157 (2009) · Zbl 1193.62021
[25] McLachlan, G; Peel, D, Contribution to the discussion of paper by S. Richardson and P.J. Green, J. R. Stat. Soc., 59, 772-773, (1997)
[26] McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, New York (2000) · Zbl 0963.62061
[27] Neykov, N., Filzmoser, P., Dimova, R., Neytchev, P.: Robust fitting of mixtures using the trimmed likelihood estimator. Comput. Stat. Data Anal. 17, 299-308 (2007) · Zbl 1328.62033
[28] Richardson, S; Green, PJ, On the Bayesian analysis of mixtures with an unknown number of components (with discussion), J. R. Stat. Soc., 59, 731-792, (1997) · Zbl 0891.62020
[29] Roeder, K, Density estimation with confidence sets exemplified by superclusters and voids in galaxies, J. Am. Stat. Assoc., 85, 617-624, (1990) · Zbl 0704.62103
[30] Titterington, D.M., Smith, A.F., Makov, U.E.: Statistical Analysis of Finite Mixture Distributions. Wiley, New York (1985) · Zbl 0646.62013
[31] Van der Vaart, A.W., Wellner, J.A.: Weak Convergence and Empirical Processes. Wiley, New York (1996) · Zbl 0862.60002
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.