Stochastic versions of the EM algorithm: An experimental study in the mixture case. (English) Zbl 0907.62024

Summary: We compare three different stochastic versions of the EM algorithm: The Stochastic EM algorithm (SEM), the “Simulated Annealing” EM algorithm (SAEM) and the Monte Carlo EM algorithm (MCEM). We focus particularly on the mixture of distributions problem. In this context, we investigate the practical behaviour of these algorithms through intensive Monte Carlo numerical simulations and a real data study. We show that, for some particular mixture situations, the SEM algorithm is almost always preferable to the EM and “simulated annealing” versions SAEM and MCEM. For some severely overlapping mixtures, however, none of these algorithms can be confidently used. Then, SEM can be used as an efficient data exploratory tool for locating significant maxima of the likelihood function. In the real data case, we show that the SEM stationary distribution provides a contrasted view of the loglikelihood by emphasizing sensible maxima.


62F10 Point estimation
65C05 Monte Carlo methods
65C99 Probabilistic methods, stochastic differential equations
Full Text: DOI


[1] DOI: 10.2307/2347474
[2] DOI: 10.1016/0304-4149(94)90149-X · Zbl 0819.60033
[3] Broniatowski M., Data Analysis and Informatics 3 pp 359– (1993)
[4] Celeux G., Computational Statistics Quaterly 2 pp 73– (1985)
[5] Celeux G., Rapport de recherche INRIA 563 (1986)
[6] Celeux G., Classification and related methods of Data Analysis pp 617– (1987)
[7] Celeux G., Cahiers du CERO 32 pp 135– (1991)
[8] Celeux G., Stochastics and Stochastics Reports 41 pp 119– (1992)
[9] DOI: 10.1080/00949659308811525
[10] Chauveau D., Revue de Statistique Appliquée 40 pp 67– (1992)
[11] Chaveau D., A Stochastic EM Algorithm for Mixtures with Censored Data, J. Statist. Plann. Inference 46 pp 1– (1995)
[12] Dempster A. P., Journal of the Royal Statis- tical Society B 39 pp 1– (1977)
[13] DOI: 10.1080/15326349308807283 · Zbl 0783.62021
[14] DOI: 10.2307/2347733 · Zbl 0707.62121
[15] Van Laarhoven P. J. M., Theory and Applications (1987) · Zbl 0643.65028
[16] Lavielle, M. and Moulines, E. 1995. ”Stochastic Approximation Version of The EM Algorithm”. Université Paris-Sud. preprint, submitted to JRSS B · Zbl 0932.62094
[17] Louis T. A., J. R. Statist. Soc. 2 pp 225– (1982)
[18] McLachlan, G. J. and Basford, K. E. 1989. ”Mixture models-inference and Applications to Clustering”. New York: Marcel Dekker. · Zbl 0697.62050
[19] DOI: 10.1098/rsta.1991.0132 · Zbl 0746.62086
[20] DOI: 10.1137/1026034 · Zbl 0536.62021
[21] Tanner M. A., Lectures Notes in Statistics 67 (1991)
[22] DOI: 10.2307/2289457 · Zbl 0619.62029
[23] DOI: 10.1080/02331889008802274 · Zbl 0714.62023
[24] Titterington D. M., Statistical analysis of Fi- nite Mixture Distribution (1985)
[25] DOI: 10.2307/2290005
[26] DOI: 10.1214/aos/1176346060 · Zbl 0517.62035
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.