The generalized linear mixed cluster-weighted model.

*(English)*Zbl 1331.62310
J. Classif. 32, No. 1, 85-113 (2015); erratum ibid. 32, No. 2, 327-355 (2015).

Summary: Cluster-weighted models (CWMs) are a flexible family of mixture models for fitting the joint distribution of a random vector composed of a response variable and a set of covariates. CWMs act as a convex combination of the products of the marginal distribution of the covariates and the conditional distribution of the response given the covariates. In this paper, we introduce a broad family of CWMs in which the component conditional distributions are assumed to belong to the exponential family and the covariates are allowed to be of mixed-type. Under the assumption of Gaussian covariates, sufficient conditions for model identifiability are provided. Moreover, maximum likelihood parameter estimates are derived using the EM algorithm. Parameter recovery, classification assessment, and performance of some information criteria are investigated through a broad simulation design. An application to real data is finally presented, with the proposed model outperforming other well-established mixture-based approaches.

##### MSC:

62H30 | Classification and discrimination; cluster analysis (statistical aspects) |

62J12 | Generalized linear models (logistic models) |

##### Keywords:

cluster-weighted models; model-based clustering; generalized linear models; mixed-type data
PDF
BibTeX
XML
Cite

\textit{S. Ingrassia} et al., J. Classif. 32, No. 1, 85--113 (2015; Zbl 1331.62310)

Full Text:
DOI

**OpenURL**

##### References:

[1] | AITKEN, A.C. (1926), “On Bernoulli’s Numerical Solution of Algebraic Equations”, in Proceedings of the Royal Society of Edinburgh, Vol. 46, pp. 289-305. · JFM 52.0098.05 |

[2] | AKAIKE, H; Petrov, BN (ed.); Csaki, F (ed.), Information theory and an extension of maximum likelihood principle, 267-281, (1973), Budapest |

[3] | BAGNATO, L; PUNZO, A, Finite mixtures of unimodal beta and gamma densities and the \(k\)-bumps algorithm, Computational Statistics, 28, 1571-1597, (2013) · Zbl 1306.65024 |

[4] | BAGNATO, L; GRESELIN, F; PUNZO, A, On the spectral decomposition in normal discriminant analysis, Communications in Statistics - Simulation and Computation, 43, 1471-1489, (2014) · Zbl 1333.62056 |

[5] | BANFIELD, JD; RAFTERY, AE, Model-based gaussian and non-Gaussian clustering, Biometrics, 49, 803-821, (1993) · Zbl 0794.62034 |

[6] | Bhattacharyya, A, On a measure of divergence between two statistical populations defined by their probability distributions, Bulletin of the Calcutta Mathematical Society, 35, 99-109, (1943) · Zbl 0063.00364 |

[7] | BIERNACKI, C; CELEUX, G; GOVAERT, G, Assessing a mixture model for clustering with the integrated completed likelihood, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 719-725, (2000) |

[8] | BIERNACKI, C; CELEUX, G; GOVAERT, G, Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models, Computational Statistics and Data Analysis, 41, 561-575, (2003) · Zbl 1429.62235 |

[9] | Bozdogan, H, Model selection and akaikes’s information criterion (AIC): the general theory and its analytical extensions, Psychometrika, 52, 345-370, (1987) · Zbl 0627.62005 |

[10] | BOZDOGAN, H, Theory & methodology of time series analysis, No. 1, (1994), Dordrecht |

[11] | DEMPSTER, AP; LAIRD, NM; RUBIN, DB, Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society. Series B (Methodological), 39, 1-38, (1977) · Zbl 0364.62022 |

[12] | FOLLMANN, DA; LAMBERT, D, Identifiability of finite mixtures of logistic regression models, Journal of Statistical Planning and Inference, 27, 375-381, (1991) · Zbl 0717.62061 |

[13] | FONSECA, JRS, The application of mixture modeling and information criteria for discovering patterns of coronary heart disease, Journal of Applied Quantitative Methods, 3, 292-303, (2008) |

[14] | FONSECA, JRS, On the performance of information criteria in latent segment models, World Academy of Science, Engineering and Technology, 63, 2010, (2010) |

[15] | FONSECA, JRS; CARDOSO, MGMS, Retail clients latent segments, 348-358, (2005), Berlin Heidelberg |

[16] | FRÜHWIRTH-SCHNATTER, S. (2006), Finite Mixture and Markov Switching Models, New York: Springer. · Zbl 1108.62002 |

[17] | Gershenfeld, N, Nonlinear inference and cluster-weighted modeling, Annals of the New York Academy of Sciences, 808, 18-24, (1997) |

[18] | GERSHENFELD, N. (1999), The Nature of Mathematical Modelling, Cambridge: Cambridge University Press. · Zbl 0905.00015 |

[19] | GERSHENFELD, N; SCHÖNER, B; METOIS, E, Cluster-weighted modelling for time-series analysis, Nature, 397, 329-332, (1999) |

[20] | GRESELIN, F; PUNZO, A, Closed likelihood ratio testing procedures to assess similarity of covariance matrices, The American Statistician, 67, 117-128, (2013) |

[21] | GRÜN, B; LEISCH, F; Heumann, C (ed.), Finite mixtures of generalized linear regression models, 205-230, (2008), Heidelberg |

[22] | GRÜN, B; LEISCH, F, flexmix version 2: finitemixtures with concomitant variables and varying and constant parameters, Journal of Statistical Software, 28, 1-35, (2008) |

[23] | Hennig, C, Identifiablity of models for clusterwise linear regression, Journal of Classification, 17, 273-296, (2000) · Zbl 1017.62058 |

[24] | HENNIG, C; LIAO, TF, How to find an appropriate clustering for mixed type variables with application to socio-economic stratification, Journal of the Royal Statistical Society: Series C (Applied Statistics), 62, 1-25, (2013) |

[25] | HURVICH, CM; TSAI, CL, Regression and time series model selection in small samples, Biometrika, 76, 297-307, (1989) · Zbl 0669.62085 |

[26] | HWANG, H; MALHOTRA, NK; KIM, Y; TOMIUK, MA; HONG, S, A comparative study on parameter recovery of three approaches to structural equation modeling, Journal of Marketing Research, 47, 699-712, (2010) |

[27] | INGRASSIA, S; MINOTTI, SC; VITTADINI, G, Local statistical modeling via the cluster-weighted approach with elliptical distributions, Journal of Classification, 29, 363-401, (2012) · Zbl 1360.62335 |

[28] | INGRASSIA, S; MINOTTI, SC; PUNZO, A, Model-based clustering via linear cluster-weighted models, Computational Statistics and Data Analysis, 71, 159-182, (2014) · Zbl 06975380 |

[29] | KARLIS, D; XEKALAKI, E, Choosing initial values for the EM algorithm for finite mixtures, Computational Statistics and Data Analysis, 41, 577-590, (2003) · Zbl 1429.62082 |

[30] | MAZZA, A., PUNZO, A., and INGRASSIA, S. (2013), {\bfflexCWM}: Flexible Cluster-Weighted Modeling, available at http://cran.fhcrc.org/web/packages/flexCWM/index.html. |

[31] | MCCULLAGH, P., and NELDER, J.A. (2000), Generalized Linear Models (2nd ed.), Boca Raton: Chapman and Hall. |

[32] | MCLACHLAN, GJ, On the EM algorithm for overdispersed count data, Statistical Methods in Medical Research, 6, 76-98, (1997) |

[33] | MCLACHLAN, G.J., and PEEL, D. (2000), Finite Mixture Models, New York: John Wiley and Sons. · Zbl 0963.62061 |

[34] | MCNICHOLAS, PD; MURPHY, TB; MCDAID, AF; FROST, D, Serial and parallel implementations of model-based clustering via parsimonious Gaussian mixture models, Computational Statistics and Data Analysis, 54, 711-723, (2010) · Zbl 1464.62131 |

[35] | MCQUARRIE, A; SHUMWAY, R; TSAI, CL, The model selection criterion aicu, Statistics and Probability Letters, 34, 285-292, (1997) · Zbl 1064.62541 |

[36] | Punzo, A, Flexible mixture modeling with the polynomial Gaussian cluster-weighted model, Statistical Modelling, 14, 257-291, (2014) |

[37] | R CORE TEAM (2013), R: A Language and Environment for Statistical Computing, Vienna, Austria: R Foundation for Statistical Computing. |

[38] | SCHÖNER, B. (2000), “Probabilistic Characterization and Synthesis of Complex Data Driven Systems”, Technical Report, Ph.D. Thesis, MIT, Cambridge. · Zbl 1333.62056 |

[39] | SCHÖNER, B; GERSHENFELD, N; Mees, A (ed.), Cluster weighted modeling: probabilistic time series prediction, characterization, and synthesism”, 365-385, (2001), Boston |

[40] | Schwarz, G, Estimating the dimension of a model, The Annals of Statistics, 6, 461-464, (1978) · Zbl 0379.62005 |

[41] | SUBEDI, S; PUNZO, A; INGRASSIA, S; MCNICHOLAS, PD, Clustering and classification via cluster-weighted factor analyzers, Advances in Data Analysis and Classification, 7, 5-40, (2013) · Zbl 1271.62137 |

[42] | Teicher, H, Identifiability of finite mixtures, Annals of Mathematical Statistics, 34, 1265-1269, (1963) · Zbl 0137.12704 |

[43] | TITTERINGTON, D.M., SMITH, A.F.M., and MAKOV, U.E. (1985), Statistical Analysis of Finite Mixture Distributions, New York: John Wiley and Sons. · Zbl 0646.62013 |

[44] | TSANAS, A; XIFARA, A, Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools, Energy and Buildings, 49, 560-567, (2012) |

[45] | VERMUNT, JK; MAGIDSON, J; Hagenaars, JA (ed.); McCutcheon, AL (ed.), Latent class cluster analysis”, 89-106, (2002), Cambridge |

[46] | WANG, P. (1994), “Mixed Regression Models for Discrete Data”, Technical Report, Ph.D. Thesis, University of British Columbia, Vancouver. |

[47] | WANG, P; PUTERMAN, ML; COCKBURN, ML; LE, ND, Mixed Poisson regression models with covariate dependent rates, Biometrics, 52, 381-400, (1996) · Zbl 0875.62407 |

[48] | Wedel, M, Concomitant variables in finite mixture models, Statistica Neerlandica, 56, 362-375, (2002) · Zbl 1076.62531 |

[49] | WEDEL, M; SARBO, W, A mixture likelihood approach for generalized linear models, Journal of Classification, 12, 21-55, (1995) · Zbl 0825.62611 |

[50] | WEDEL, M., and KAMAKURA, W.A. (2001), Market Segmentation: Conceptual and Methodological Foundations (2nd ed.), Boston MA: Kluwer Academic Publishers. · Zbl 1293.62261 |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.