×

Maximum likelihood estimation of mixtures of factor analyzers. (English) Zbl 1464.62134

Summary: Mixtures of factor analyzers have been receiving wide interest in statistics as a tool for performing clustering and dimension reduction simultaneously. In this model it is assumed that, within each component, the data are generated according to a factor model. Therefore, the number of parameters on which the covariance matrices depend is reduced. Several estimation methods have been proposed for this model, both in the classical and in the Bayesian framework. However, so far, a direct maximum likelihood procedure has not been developed. This direct estimation problem, which simultaneously allows one to derive the information matrix for the mixtures of factor analyzers, is solved. The effectiveness of the proposed procedure is shown on a simulation study and on a toy example.

MSC:

62-08 Computational methods for problems pertaining to statistics
62H12 Estimation in multivariate analysis
62H25 Factor analysis and principal components; correspondence analysis
62H30 Classification and discrimination; cluster analysis (statistical aspects)

Software:

R
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Boldea, O.; Magnus, J.R., Maximum likelihood estimation of the multivariate normal mixture model, Journal of the American statistical association, 104, 1539-1549, (2009) · Zbl 1205.62065
[2] Cook, R.D.; Weisberg, S., An introduction to regression graphics, (1994), Wiley New York · Zbl 0925.62287
[3] Cramér, H., Mathematical methods of statistics, (1946), Princeton University Press Princeton, New Jersey · Zbl 0063.01014
[4] Day, N.E., Estimating the components of a mixture of two normal distributions, Biometrika, 56, 463-474, (1969) · Zbl 0183.48106
[5] Dempster, N.M.; Laird, A.P.; Rubin, D.B., Maximum likelihood from incomplete data via the EM algorithm (with discussion), Journal of the royal statistical society. series B, 39, 1-38, (1977) · Zbl 0364.62022
[6] Fokoué, E.; Titterington, D.M., Mixtures of factor analysers. Bayesian estimation and inference in stochastic simulation, Machine learning, 50, 73-94, (2003) · Zbl 1033.68085
[7] Fraley, C.; Raftery, A.E., Model-based clustering, discriminant analysis, and density estimation, Journal of the American statistical association, 97, 611-631, (2002) · Zbl 1073.62545
[8] Ghahramani, Z.; Beal, M.J., Variational inference for Bayesian mixtures of factor analyzers, (), 449-455
[9] Ghahramani, Z., Hilton, G.E., 1997. The EM algorithm for mixture of factor analyzers. Techical Report CRG-TR-96-1. Department of Computer Science, University of Toronto, Canada.; Ghahramani, Z., Hilton, G.E., 1997. The EM algorithm for mixture of factor analyzers. Techical Report CRG-TR-96-1. Department of Computer Science, University of Toronto, Canada.
[10] Louis, T.A., Finding the observed information matrix when using the EM algorithm, Journal of the royal statistical society. series B, 44, 226-233, (1982) · Zbl 0488.62018
[11] McLachlan, G.J.; Basford, K.E., Mixture models: inference and application to clustering, (1988), Marcel Dekker New York · Zbl 0697.62050
[12] McLachlan, G.J.; Peel, D., Finite mixture models, (2000), John Wiley & Sons Inc. New York · Zbl 0963.62061
[13] McLachlan, G.J.; Peel, D., Mixtures of factor analyzers, (), 599-606
[14] McLachlan, G.J.; Peel, D.; Bean, R.W., Modelling high-dimensional data by mixtures of factor analyzers, Computational statistics and data analysis, 41, 379-388, (2003) · Zbl 1256.62036
[15] Meng, X.L.; Rubin, D.B., Using EM to obtain asymptotic variance – covariance matrices: the SEM algorithm, Journal of the American statistical association, 86, 899-909, (1991)
[16] Meng, X.L.; van Dyk, D., The EM algorithm—an old folk song sung to a fast new tune (with discussion), Journal of the royal statistical society. series B (methodological), 59, 511-567, (1997) · Zbl 1090.62518
[17] R Development Core Team 2008. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL: http://www.R-project.org; R Development Core Team 2008. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL: http://www.R-project.org
[18] Raftery, A.E.; Dean, N., Variable selection for model-based clustering, Journal of the American statistical association, 101, 168-178, (2006) · Zbl 1118.62339
[19] Redner, R.A.; Walker, H.F., Mixture densities, maximum likelihood and the EM algorithm, SIAM review, 26, 195-239, (1984) · Zbl 0536.62021
[20] Stephens, M., Dealing with label-switching in mixture models, Journal of the royal statistical society, series B, 62, 795-809, (2000) · Zbl 0957.62020
[21] Utsugi, A.; Kumagai, T., Bayesian analysis of mixtures of factor analyzers, Neural computation, 13, 993-1002, (2001) · Zbl 1042.62028
[22] Wolfe, J.H., Pattern clustering by multivariate mixture analysis, Multivariate behavioral research, 5, 329-350, (1970)
[23] Zhao, J.H.; Yu, P.L.H., Fast ML estimation for the mixture of factor analyzers via an ECM algorithm, IEEE transactions on neural networks, 19, 1956-1961, (2008)
[24] Zhou, X.; Liu, X., The EM algorithm for the extended finite mixture of the factor analyzers model, Computational statistics and data analysis, 52, 3939-3953, (2008) · Zbl 1452.62428
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.