×

zbMATH — the first resource for mathematics

Ascent-based Monte Carlo expectation-maximization. (English) Zbl 1075.65011
Summary: The expectation-maximization (EM) algorithm is a popular tool for maximizing likelihood functions in the presence of missing data. Unfortunately, EM often requires the evaluation of analytically intractable and high dimensional integrals. The Monte Carlo EM (MCEM) algorithm is the natural extension of EM that employs Monte Carlo methods to estimate the relevant integrals. Typically, a very large Monte Carlo sample size is required to estimate these integrals within an acceptable tolerance when the algorithm is near convergence. Even if this sample size were known at the onset of implementation of MCEM, its use throughout all iterations is wasteful, especially when accurate starting values are not available.
We propose a data-driven strategy for controlling Monte Carlo resources in MCEM. The algorithm proposed improves on similar existing methods by recovering EM’s ascent (i.e. likelihood increasing) property with high probability, being more robust to the effect of user-defined inputs and handling classical Monte Carlo and Markov chain Monte Carlo methods within a common framework. Because of the first of these properties we refer to the algorithm as ‘ascent-based MCEM’. We apply ascent-based MCEM to a variety of examples, including one where it is used to accelerate the convergence of deterministic EM dramatically.

MSC:
65C40 Numerical analysis or methods applied to Markov chains
65C60 Computational problems in statistics (MSC2010)
65C05 Monte Carlo methods
60J22 Computational methods in Markov chains
PDF BibTeX Cite
Full Text: DOI
References:
[1] Billingsley P., Probability and Measure (1995)
[2] DOI: 10.1111/1467-9868.00176 · Zbl 0917.62058
[3] DOI: 10.1191/147108201128249 · Zbl 1102.62019
[4] DOI: 10.1093/biomet/89.4.745 · Zbl 1036.62004
[5] Christensen O. F., R News 2 pp 26– (2002)
[6] Dempster A. P., J. R. Statist. Soc. 39 pp 1– (1977)
[7] DOI: 10.1111/1467-9876.00113 · Zbl 0904.62119
[8] Geyer C. J., J. R. Statist. Soc. 56 pp 261– (1994)
[9] DOI: 10.1198/016214501753208762 · Zbl 1072.62612
[10] DOI: 10.1006/jmva.1998.1778 · Zbl 0922.60069
[11] DOI: 10.1093/biomet/89.4.731 · Zbl 1035.60080
[12] Ihaka R., J. Comput. Graph. Statist. 5 pp 299– (1996)
[13] DOI: 10.1198/1061860031338
[14] G. L. Jones, M. Haran, and B. S. Caffo (2004 ) Output analysis for Markov chain Monte Carlo simulations .Technical Report. School of Statistics, University of Minnesota, Minneapolis. · Zbl 1171.62316
[15] Jones G. L., Statist. Sci. 16 pp 312– (2001)
[16] Kendall M. G., The Advanced Theory of Statistics (1958)
[17] Lange K., Numerical Analysis for Statisticians (1999) · Zbl 0920.62001
[18] DOI: 10.1198/106186001317115045 · Zbl 04568631
[19] Littell R. C., SAS System for Mixed Models (1996)
[20] Louis T. A., J. R. Statist. Soc. 44 pp 226– (1982)
[21] McCulloch C. E., J. Am. Statist. Ass. 89 pp 330– (1994)
[22] McCulloch C. E., J. Am. Statist. Ass. 92 pp 162– (1997)
[23] Meyn S. P., Markov Chains and Stochastic Stability (1993) · Zbl 0925.60001
[24] Mykland P., J. Am. Statist. Ass. 90 pp 233– (1995)
[25] Nummelin E., General Irreducible Markov Chains and Non-negative Operators (1984) · Zbl 0551.60066
[26] DOI: 10.1137/0330046 · Zbl 0762.62022
[27] DOI: 10.1016/S0167-9473(98)00075-9 · Zbl 1042.62597
[28] Robert C. P., Monte Carlo Statistical Methods (1999) · Zbl 0935.62005
[29] Searle S. R., Variance Components (1992) · Zbl 0850.62007
[30] DOI: 10.1111/1467-9868.00334 · Zbl 1059.62059
[31] Wei G. C. G., J. Am. Statist. Ass. 85 pp 699– (1990)
[32] Wu C. F. J., Ann. Statist. 11 pp 95– (1983)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.