Variational inference for Dirichlet process mixtures. (English) Zbl 1331.62259

Blei, David M.; Jordan, Michael I.

Bayesian Anal. 1, No. 1, 121-144 (2006).

Summary: Dirichlet process (DP) mixture models are the cornerstone of nonparametric Bayesian statistics, and the development of Monte-Carlo Markov chain (MCMC) sampling methods for DP mixtures has enabled the application of nonparametric Bayesian methods to a variety of practical data analysis problems. However, MCMC sampling can be prohibitively slow, and it is important to explore alternatives. One class of alternatives is provided by variational methods, a class of deterministic algorithms that convert inference problems into optimization problems. Thus far, variational methods have mainly been explored in the parametric setting, in particular within the formalism of the exponential family. In this paper, we present a variational inference algorithm for DP mixtures. We present experiments that compare the algorithm to Gibbs sampling algorithms for DP mixtures of Gaussians and present an application to a large-scale image analysis problem.

MSC:

62G32	Statistics of extreme values; tail inference
62G09	Nonparametric statistical resampling methods
94A08	Image processing (compression, reconstruction, etc.) in information and communication theory

Keywords:

Dirichlet processes; hierarchical models; variational inference; image processing; Bayesian computation