Approximations for binary Gaussian process classification. (English) Zbl 1225.62087

Nickisch, Hannes; Rasmussen, Carl Edward

J. Mach. Learn. Res. 9, 2035-2078 (2008).

Summary: We provide a comprehensive overview of many recent algorithms for approximate inference in Gaussian process models for probabilistic binary classification. The relationships between several approaches are elucidated theoretically, and the properties of the different algorithms are corroborated by experimental results. We examine both 1) the quality of the predictive distributions and 2) the suitability of the different marginal likelihood approximations for model selection (selecting hyperparameters) and compare them to a gold standard based on MCMC. Interestingly, some methods produce good predictive distributions although their marginal likelihood approximations are poor. Strong conclusions are drawn about the methods: The Expectation Propagation algorithm is almost always the method of choice unless the computational budget is very tight. We also extend existing methods in various ways, and provide unifying code implementing all approaches.

MSC:

62H30	Classification and discrimination; cluster analysis (statistical aspects)
65C40	Numerical analysis or methods applied to Markov chains
68T05	Learning and adaptive systems in artificial intelligence

Keywords:

Gaussian process priors; probabilistic classification; Laplace approximation; expectation propagation; variational bounding; mean field methods; marginal likelihood evidence; MCMC