Estimation and selection for the latent block model on categorical data. (English) Zbl 1331.62149

Summary: This paper deals with estimation and model selection in the Latent Block Model (LBM) for categorical data. First, after providing sufficient conditions ensuring the identifiability of this model, we generalise estimation procedures and model selection criteria derived for binary data. Secondly, we develop Bayesian inference through Gibbs sampling and with a well calibrated non informative prior distribution, in order to get the MAP estimator: this is proved to avoid the traps encountered by the LBM with the maximum likelihood methodology. Then model selection criteria are presented. In particular an exact expression of the integrated completed likelihood criterion requiring no asymptotic approximation is derived. Finally numerical experiments on both simulated and real data sets highlight the appeal of the proposed estimation and model selection procedures.


62F15 Bayesian inference
62H12 Estimation in multivariate analysis
62H30 Classification and discrimination; cluster analysis (statistical aspects)
Full Text: DOI Link


[1] Allman, E., Mattias, C., Rhodes, J.: Identifiability of parameters in latent structure models with many observed variables. Ann. Stat. 37, 3099-3132 (2009) · Zbl 1191.62003
[2] Banerjee, A; Dhillon, I; Ghosh, J; Merugu, S; Modha, DS, A generalized maximum entropy approach to Bregman co-clustering and matrix approximation, J. Mach. Learn. Res., 8, 1919-1986, (2007) · Zbl 1222.68139
[3] Baudry, J.-P. : Sélection de modèle pour la classification non supervisée. Choix du nombre de classes. PhD thesis, Université Paris Sud, December 2009.
[4] Baudry, J.-P., Raftery, A.E., Celeux, G., Lo, K., Gottardo, R.: Combining mixture components for clustering. J. Comput. Gr. Stat. 19, 332-353 (2010)
[5] Biernacki, C., Celeux, G., Govaert, G.: Assessing a mixture model for clustering with the integrated completed likelihood. IEEE Trans. Pattern Anal. Mach. Intell. 22, 719-725 (Jul 2000)
[6] Carreira-Perpiñàn, M; Renals, S, Practical identifiability of finite mixtures of multivariate Bernoulli distributions, Neural Comput., 12, 141-152, (2000)
[7] Celeux, G; Diebolt, J, Stochastic versions of the EM algorithm, Comput. Stat. Quat., 2, 73-82, (1985)
[8] Celisse, A; Daudin, J-J; Latouche, P, Consistency of maximum-likelihood and variational estimators in the stochastic block model, Electron. J. Stat., 6, 1847-1899, (2012) · Zbl 1295.62028
[9] Dempster, AP; Laird, NM; Rubin, DB, Maximum likelihood from incomplete data via the EM algorithm (with discussion), J. R. Stat. Soc. Ser. B, 39, 1-38, (1977) · Zbl 0364.62022
[10] Frühwirth-Schnatter, S.: Finite Mixture and Markov Switching Models. Springer series in statistics, Springer (2006) · Zbl 1108.62002
[11] Frühwirth-Schnatter, S.: Mixtures : Estimation and Applications, Chapter Dealing with Label Switching Under Model Uncertainty. Wiley, Chichester (2011)
[12] Govaert, G. : Algorithme de classification d’un tableau de contingence. In First international Symposium on Data Analysis and Informatics, pp. 487-500, Versailles, 1977. INRIA. · Zbl 1452.62444
[13] Govaert, G. : Classification croisée. PhD thesis, Université Paris 6, France, 1983. · Zbl 0817.92002
[14] Govaert, G; Nadif, M, Block clustering with Bernoulli mixture models: comparison of different approaches, Comput. Stat. Data Anal., 52, 3233-3245, (2008) · Zbl 1452.62444
[15] Govaert, G; Nadif, M, Latent block model for contingency table, Commun. Stat. Theory Methods, 39, 416-425, (2010) · Zbl 1187.62117
[16] Gyllenberg, M; Koski, T; Reilink, E; Verlann, M, Non-uniqueness in probabilistic numerical identification of bacteria, J. Appl. Probab., 31, 542-548, (1994) · Zbl 0817.92002
[17] Jagalur, M; Pal, C; Learned-Miller, E; Zoeller, RT; Kulp, D, Analyzing in situ gene expression in the mouse brain with image registration, feature extraction and block clustering, BMC Bioinform., 8, s5, (2007)
[18] Keribin, C.: Consistent estimation of the order of mixture models. Sankhya Ser. A 62, 49-66 (2000) · Zbl 1081.62516
[19] Keribin, C, Méthodes bayésiennes variationnelles: concepts et applications en neuroimagerie, Journal de la Société Française de Statistique, 151, 107-131, (2010) · Zbl 1316.62041
[20] Keribin, C., Brault, V., Celeux, G., Govaert, G.: Model selection for the binary latent block model. Proceedings of COMPSTAT 2012, 2012. · Zbl 1331.62149
[21] Keribin, C., Brault, V., Celeux, G., Govaert, G. : Estimation and Selection for the Latent Block Model on Categorical Data. Rapport de recherche RR-8264, INRIA, March 2013. URL http://hal.inria.fr/hal-00802764 · Zbl 1331.62149
[22] Lomet, A.: Sélection de modèle pour la classification croisée de données continues. PhD thesis, Université de Technologie de Compiègne, December 2012. · Zbl 1322.62046
[23] Lomet, A., Govaert, G., Grandvalet, Y.: Un protocole de simulation de données pour la classification croisée. In 44ème journées de statistique, Bruxelles, Mai 2012. · Zbl 1228.62034
[24] Madeira, SC; Oliveira, AL, Biclustering algorithms for biological data analysis: a survey, IEEE/ACM Trans. Comput. Biol. Bioinf., 1, 24-45, (2004)
[25] Mariadassou, M., Matias, C.: Convergence of the groups posterior distribution in latent or stochastic block models. arXiv, preprint arXiv:1206.7101v2, 2013. · Zbl 1329.62285
[26] McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extensions, 2nd edn. Wiley, Nex York (2008) · Zbl 1165.62019
[27] McLachlan, G.J., Peel, D.: Finite Mixture Models, 2nd edn. Wiley, Nex York (2000) · Zbl 0963.62061
[28] Meeds, E, Roweis, S: Nonparametric bayesian biclustering. Technical Report UTML TR 2007-001, Department of Computer Science, University of Toronto, 2007. · Zbl 0379.62005
[29] Rousseau, J; Mengersen, K, Asymptotic behaviour of the posterior distribution in overfitted models, J. Roy. Stat. Soc., 73, 689-710, (2011) · Zbl 1228.62034
[30] Schwarz, G, Estimating the dimension of a model, Ann. Stat., 6, 461-464, (1978) · Zbl 0379.62005
[31] Shan, H., Banerjee, A.: Bayesian co-clustering. In Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, ICDM ’08, pp. 530-539, Washington, DC, 2008. IEEE Computer Society.
[32] Wyse, J; Friel, N, Block clustering with collapsed latent block models, Stat. Comput., 22, 415-428, (2012) · Zbl 1322.62046
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.