Mixtures of general location model with factor analyzer covariance structure for clustering mixed type data. (English) Zbl 07480917

Summary: Cluster analysis is one of the most widely used method in statistical analyses, in which homogeneous subgroups are identified in a heterogeneous population. Due to the existence of the continuous and discrete mixed data in many applications, so far, some ordinary clustering methods such as, hierarchical methods, \(k\)-means and model-based methods have been extended for analysis of mixed data. However, in the available model-based clustering methods, by increasing the number of continuous variables, the number of parameters increases and identifying as well as fitting an appropriate model may be difficult. In this paper, to reduce the number of the parameters, for the model-based clustering mixed data of continuous (normal) and nominal data, a set of parsimonious models is introduced. Models in this set are extended, using the general location model approach, for modeling distribution of mixed variables and applying factor analyzer structure for covariance matrices. The ECM algorithm is used for estimating the parameters of these models. In order to show the performance of the proposed models for clustering, results from some simulation studies and analyzing two real data sets are presented.


62Pxx Applications of statistics
Full Text: DOI


[1] Ahmad, A.; Dey, L., A k-mean clustering algorithm for mixed numeric and categorical data, Data Knowledge Eng., 63, 503-527 (2007)
[2] Amiri, L.; Khazaei, M.; Ganjali, M., General location model with factor analyzer covariance matrix structure and its applications, Adv. Data. Anal. Classif., 11, 593-609 (2017) · Zbl 1414.62205
[3] An, X.; Bentler, P., Extended mixture factor analysis model with covariates for mixed binary and continuous responses, Stat. Med., 30, 2634-2647 (2011)
[4] Azzalini, A.; Menardi, G., Clustering via nonparametric density estimation: The R package pdfCluster, J. Stat. Softw., 57, 1-26 (2014)
[5] Baek, J.; McLachlan, G. J., Mixtures of common t-factor analyzers for clustering high-dimensional microarray data, Bioinformatics, 27, 1269-1276 (2011)
[6] Baek, J.; McLachlan, G. J.; Flack, L. K., Mixtures of factor analyzers with common factor loadings: Applications to the clustering and visualisation of high-dimensional data, IEEE. Trans. Pattern. Anal. Mach. Intell., 32, 1298-1309 (2010)
[7] Banfield, J. D.; Raftery, A. E., Model-based Gaussian and non-Gaussian clustering, Biometrics, 49, 803-821 (1993) · Zbl 0794.62034
[8] Becker, C.; Fried, R.; Kuhnt, S., Robustness and Complex Data Structures (2013), Springer-Verlag: Springer-Verlag, Berlin, Heidelberg
[9] Browne, R. P.; McNicholas, P. D., Model-based clustering and classification of data with mixed type, J. Statist. Plann. Inference, 142, 2976-2984 (2012) · Zbl 1335.62093
[10] Cai, J. H.; Song, X. Y.; Lam, K. H.; Ip, H. S., A mixture of generalized latent variable models for mixed mode and heterogeneous data, Comput. Statist. Data Anal., 55, 2889-2907 (2011) · Zbl 1218.62012
[11] Celeux, G.; Govaert, G., Gaussian parsimonious clustering models, Pattern Recognit., 28, 781-793 (1995)
[12] Everitt, B. S., A finite mixture model for the clustering of mixed mode data, Stat. Probab. Lett., 6, 305-309 (1988)
[13] Everitt, B. S.; Landau, S.; Leese, M.; Stahl, D., Cluster Analysis (2011), John Wiley and Sons: John Wiley and Sons, Chichester · Zbl 1274.62003
[14] Everitt, B. S.; Merette, C., The clustering of mixed-mode data: A comparison of possible approaches, J. Appl. Stat., 17, 283-297 (1990)
[15] Foss, A. and Markatou, M., Methods for Clustering Mixed-Type Data, R package version, 2016. http://CRAN.R-project.org/package=kamila · Zbl 1432.62182
[16] Foss, A.; Markatou, M.; Ray, B.; Heching, A., A semiparametric method for clustering mixed data, Mach. Learn., 105, 419-458 (2016) · Zbl 1432.62182
[17] Fraley, C.; Raftery, A. E., Model-based clustering, discriminant analysis, and density estimation, J. Amer. Statist. Assoc., 97, 611-631 (2002) · Zbl 1073.62545
[18] Gan, G.; Ma, C.; Wu, J., Data Clustering: Theory, Algorithms, and Applications (2007), ASA-SIAM: ASA-SIAM, Philadelphia
[19] Gower, J. C., A general coefficient of similarity and some of its properties, Biometrics, 27, 857-872 (1971)
[20] Hennig, C.; Liao, T. F., How to find an appropriate clustering for mixed-type variables with application to socio-economic stratification, J. R. Stat. Soc. Ser. C, 62, 309-369 (2013)
[21] Hennig, C.; Meila, M.; Murtagh, F.; Rocci, R., Handbook of Cluster Analysis (2015), Chapman and Hall, CRC Press: Chapman and Hall, CRC Press, Boca Raton, FL
[22] Hubert, L.; Arabie, P., Comparing partitions, J. Classif., 2, 193-218 (1985)
[23] Kaufman, L.; Rousseeuw, P. J., Finding Groups in Data. An Introduction to Cluster Analysis (1990), John Wiley and Sons Inc.: John Wiley and Sons Inc., New York · Zbl 1345.62009
[24] Lawrence, C.; Krzanowski, W., Mixture separation for mixed-mode data, Stat. Comput., 6, 85-92 (1996)
[25] Mardin, C.; Hothorn, T.; Peters, A.; Jünemann, A.; Michelson, G.; Lausen, B., New glaucoma classification method based on standard HRT parameters by bagging classification trees, J. Glaucoma, 12, 340-346 (2003)
[26] McLachlan, G. J.; Krishnan, T., The EM Algorithm and Extensions (2008), John Wiley and Sons: John Wiley and Sons, New Jersey · Zbl 1165.62019
[27] McLachlan, G. J.; Peel, D., Finite mixture models (2000), John Wiley and Sons: John Wiley and Sons, New York · Zbl 0963.62061
[28] McLachlan, G. J.; Peel, D.; Bean, R. W., Modelling high-dimensional data by mixtures of factor analyzers, Comput. Statist. Data Anal., 41, 379-388 (2003) · Zbl 1256.62036
[29] McNicholas, P. D., Mixture Model-based Classification (2017), Chapman and Hall/CRC Press: Chapman and Hall/CRC Press, Boca Raton, FL · Zbl 1454.62005
[30] McNicholas, P. D.; Murphy, T. B., Parsimonious Gaussian mixture models, Stat. Comput., 18, 285-296 (2008)
[31] McParland, D.; Gormley, I. C., Model based clustering for mixed data: ClustMD, Adv. Data Anal. Classif., 10, 155-169 (2016) · Zbl 1414.62254
[32] Meng, X. L.; Rubin, D., Maximum likelihood estimation via the ECM algorithm: A general framework, Biometrika, 80, 267-278 (1993) · Zbl 0778.62022
[33] Mengersen, K. L.; Robert, C. P.; Titterington, D. M., Mixtures, Estimation and Applications (2011), John Wiley and Sons: John Wiley and Sons, UK · Zbl 1218.62003
[34] Modha, D. S.; Spangler, W. S., Feature weighting in k-means clustering, Mach. Learn., 52, 217-237 (2003) · Zbl 1039.68111
[35] Morlini, I., A latent variables approach for clustering mixed binary and continuous variables within a gaussian mixture model, Adv. Data. Anal. Classif., 6, 5-28 (2012) · Zbl 1284.62384
[36] Olkin, I.; Tate, R. F., Multivariate correlation models with mixed discrete and continuous variables, Ann. Math. Stat., 32, 448-465 (1961) · Zbl 0113.35101
[37] Pawlowsky-Glahn, V.; Buccianti, A., Compositional Data Analysis: Theory and Applications (2011), John Wiley and Sons: John Wiley and Sons, UK · Zbl 1103.62111
[38] Peters, A., Hothorn, T., Ripley, B.D., Therneau, T., and Atkinson, B., Improved Predictors, R package version 0.9-6, 2017. http://CRAN.R-project.org/package=ipred
[39] Peters, A.; Lausen, B.; Michelson, G.; Gefeller, O., Diagnosis of glaucoma by indirect classifiers, Methods Inf. Med., 1, 99-103 (2003)
[40] Ranalli, M.; Rocci, R., Mixture models for mixed-type data through a composite likelihood approach, Comput. Statist. Data Anal., 110, 87-102 (2017) · Zbl 1466.62181
[41] Reimann, C.; Arnoldussen, A.; Boyd, R.; Finne, T-E.; Nordgulen, Ø.; Volden, T.; Englmair, P., The Influence of a city on element contents of a terrestrial moss (Hylocomium splendens), Sci. Total Environ., 369, 419-432 (2006)
[42] Reimann, C.; Arnoldussen, A.; Boyd, R.; Finne, T-E.; Koller, F.; Nordgulen, Ø.; Englmair, P., Element contents in leaves of four plant species (birch, mountain ash, fern and spruce) along anthropogenic and geogenic concentration gradients, Sci. Total Environ., 377, 416-433 (2007)
[43] Schwarz, G., Estimating the dimension of a model, Ann. Stat., 6, 461-464 (1978) · Zbl 0379.62005
[44] Subedi, S.; Punzo, A.; Ingrassia, S.; McNicholas, P. D., Clustering and classification via cluster-weighted factor analyzers, Adv. Data Anal. Classif., 7, 5-40 (2013) · Zbl 1271.62137
[45] Wall, M. M.; Guo, J.; Amemiya, Y., Mixture factor analysis for approximating a nonnormally distributed continuous latent factor with continuous and dichotomous observed variables, Multivariate Behav. Res., 47, 276-313 (2012)
[46] Willse, A.; Boik, R. J., Identifiable finite mixtures of location models for clustering mixed-mode data, Stat. Comput., 9, 111-121 (1999)
[47] Xu, R.; Wunsch II, D. C., Clustering (2009), Wiley-IEEE Press: Wiley-IEEE Press, New York
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.