Vprop swMATH ID: 22203 Software Authors: Mohammad Emtiyaz Khan, Zuozhu Liu, Voot Tangkaratt, Yarin Gal Description: Vprop: Variational Inference using RMSprop. Many computationally-efficient methods for Bayesian deep learning rely on continuous optimization algorithms, but the implementation of these methods requires significant changes to existing code-bases. In this paper, we propose Vprop, a method for Gaussian variational inference that can be implemented with two minor changes to the off-the-shelf RMSprop optimizer. Vprop also reduces the memory requirements of Black-Box Variational Inference by half. We derive Vprop using the conjugate-computation variational inference method, and establish its connections to Newton’s method, natural-gradient methods, and extended Kalman filters. Overall, this paper presents Vprop as a principled, computationally-efficient, and easy-to-implement method for Bayesian deep learning. Homepage: https://arxiv.org/abs/1712.01038 Keywords: Machine Learning; arXiv stat.ML; Learning; arXiv cs.LG; arXiv; Bayesian deep learning; Gaussian variational inference; RMSprop optimizer Related Software: Adam; RMSprop; AdaGrad; PRMLT Cited in: 1 Publication Cited by 3 Authors 1 Bach, Francis R. 1 Bonnabel, Silvère 1 Lambert, Marc Cited in 1 Serial 1 Statistics and Computing Cited in 2 Fields 1 Statistics (62-XX) 1 Systems theory; control (93-XX) Citations by Year