ADVI swMATH ID: 34040 Software Authors: Alp Kucukelbir, Rajesh Ranganath, Andrew Gelman, David M. Blei Description: Automatic Variational Inference in Stan. Variational inference is a scalable technique for approximate Bayesian inference. Deriving variational inference algorithms requires tedious model-specific calculations; this makes it difficult to automate. We propose an automatic variational inference algorithm, automatic differentiation variational inference (ADVI). The user only provides a Bayesian model and a dataset; nothing else. We make no conjugacy assumptions and support a broad class of models. The algorithm automatically determines an appropriate variational family and optimizes the variational objective. We implement ADVI in Stan (code available now), a probabilistic programming framework. We compare ADVI to MCMC sampling across hierarchical generalized linear models, nonconjugate matrix factorization, and a mixture model. We train the mixture model on a quarter million images. With ADVI we can use variational inference on any model we write in Stan. Homepage: https://arxiv.org/abs/1506.03431 Dependencies: Stan Keywords: Machine Learning; arXiv_stat.ML; Stan; Variational Inference; Bayesian inference; probabilistic programming; Automatic Related Software: Stan; PRMLT; NUTS; R; BayesDA; ADADELTA; TensorFlow; RStan; Edward; PyTorch; Adam; RMSprop; AdaGrad; DiffSharp; rstan; aplore3; bfa; JAGS; Theano; Pyro Cited in: 35 Publications Standard Articles 2 Publications describing the Software, including 1 Publication in zbMATH Year Automatic differentiation variational inference. Zbl 1437.62109Kucukelbir, Alp; Tran, Dustin; Ranganath, Rajesh; Gelman, Andrew; Blei, David M. 2017 Automatic Variational Inference in Stan Alp Kucukelbir, Rajesh Ranganath, Andrew Gelman, David M. Blei 2015 all top 5 Cited by 111 Authors 6 Nott, David John 4 Blei, David M. 3 Smith, Michael Stanley 2 Gelman, Andrew 2 Loaiza-Maya, Rubén 2 Ong, Victor M. H. 2 Srivastava, Sanvesh 2 Tran, Minh Ngoc 2 Wang, Yixin 1 Aastveit, Knut Are 1 Akbayrak, Semih 1 Aliverti, Emanuele 1 Andrade, Daniel 1 Azzimonti, Laura 1 Bakka, Haakon C. 1 Balasubramanian, Krishnakumar 1 Barajas-Solano, David A. 1 Baydin, Atılım Güneş 1 Belitser, Eduard 1 Beskos, Alexandros 1 Bharath, Karthik 1 Brinkerhoff, Douglas J. 1 Broderick, Tamara 1 Castro-Camilo, Daniela 1 Clarke, Bertrand S. 1 Clyde, Merlise A. 1 Corani, Giorgio 1 Dauwels, Justin 1 de Heide, Rianne 1 De Iorio, Maria 1 De Vries, Bert J. M. 1 Drori, Iddo 1 Drovandi, Christopher C. 1 Dunson, David Brian 1 Duraisamy, Karthik 1 Englert, Berthold-Georg 1 Evans, Michael John 1 Fearnhead, Paul 1 Ferreira, Marco A. R. 1 Franck, Christopher T. 1 Franco-Villoria, Maria 1 Freni-Sterrantino, Anna 1 Fukumizu, Kenji 1 Galeano, Pedro 1 Ghadimi, Saeed 1 Giordano, Ryan 1 Grünwald, Peter D. 1 Grushka-Cockayne, Yael 1 Hao, Jie 1 Hoffman, Matthew D. 1 Hoogerheide, Lennart F. 1 Huser, Raphaël 1 Iacopini, Matteo 1 Jordan, Michael Irwin 1 Jose, Victor Richmond R. 1 Karimi, Belhal 1 Kelter, Riko 1 Kohn, Robert J. 1 Kucukelbir, Alp 1 Kurtek, Sebastian 1 Lavielle, Marc 1 Li, Cheng 1 Li, Meng 1 Lichtendahl, Kenneth C. jun. 1 Longtin, André 1 Macke, Jakob H. 1 MacNab, Ying Cai 1 Mandt, Stephan 1 McAlinn, Kenichiro 1 McLean, Mathew W. 1 Moulines, Eric 1 Nemeth, Christopher 1 Nurushev, Nurzhan 1 Opitz, Thomas 1 Pan, Shaowu 1 Pearlmutter, Barak A. 1 Pericchi, Luis Raul 1 Radul, Alexey Andreyevich 1 Ranganath, Rajesh 1 René, Alexandre 1 Roy, Abhishek 1 Rue, Håvard 1 Russo, Massimiliano 1 Saha, Abhijoy 1 Sarı, Alp 1 Şenöz, İsmail 1 Shen, Lingqing 1 Shin, Minsuk 1 Simpson, Daniel Peter 1 Siskind, Jeffrey Mark 1 Sisson, Scott A. 1 Takeda, Akiko 1 Tan, Linda S. L. 1 Tartakovsky, Alexandre M. 1 Tonellato, Stefano F. 1 Tran, Dustin 1 van Dijk, Herman K. 1 Vehtari, Aki 1 Wand, Matthew P. 1 Wang, Xueou ...and 11 more Authors all top 5 Cited in 14 Serials 6 Journal of Machine Learning Research (JMLR) 6 Journal of Computational and Graphical Statistics 4 Journal of the American Statistical Association 4 Statistics and Computing 3 Computational Statistics and Data Analysis 2 Journal of Computational Physics 2 Bayesian Analysis 1 Journal of Mathematical Psychology 1 Statistical Science 1 International Journal of Approximate Reasoning 1 Neural Computation 1 Test 1 Bernoulli 1 SIAM Journal on Applied Dynamical Systems all top 5 Cited in 11 Fields 30 Statistics (62-XX) 7 Numerical analysis (65-XX) 5 Computer science (68-XX) 4 Operations research, mathematical programming (90-XX) 1 Partial differential equations (35-XX) 1 Dynamical systems and ergodic theory (37-XX) 1 Operator theory (47-XX) 1 Fluid mechanics (76-XX) 1 Geophysics (86-XX) 1 Game theory, economics, finance, and other social and behavioral sciences (91-XX) 1 Biology and other natural sciences (92-XX) Citations by Year