FFJORD swMATH ID: 34244 Software Authors: Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, Ilya Sutskever, David Duvenaud Description: FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models. A promising class of generative models maps points from a simple distribution to a complex distribution through an invertible neural network. Likelihood-based training of these models requires restricting their architectures to allow cheap computation of Jacobian determinants. Alternatively, the Jacobian trace can be used if the transformation is specified by an ordinary differential equation. In this paper, we use Hutchinson’s trace estimator to give a scalable unbiased estimate of the log-density. The result is a continuous-time invertible generative model with unbiased density estimation and one-pass sampling, while allowing unrestricted neural network architectures. We demonstrate our approach on high-dimensional density estimation, image generation, and variational inference, achieving the state-of-the-art among exact likelihood methods with efficient sampling. Homepage: https://arxiv.org/abs/1810.01367 Related Software: torchdiffeq; Glow; ANODEs; PyTorch; MADE; PixelCNN++; WaveGlow; Adam; FloWaveNet; Flow++; UCI-ml; nflows; nsf; NICE; Pyro; GitHub; TensorFlow; DifferentialEquations.jl; DiffEqFlux; torchsde Cited in: 8 Publications all top 5 Cited by 27 Authors 1 Arenz, Oleg 1 Brehmer, Johann 1 Cichocki, Andrzej 1 Cranmer, Kyle 1 Cuchiero, Christa 1 Daulbaev, Talgat 1 Drori, Iddo 1 Gusak, Julia 1 Koutsourelakis, Phaedon-Stelios 1 Lakshminarayanan, Balaji 1 Larsson, Martin 1 Louppe, Gilles 1 Marzouk, Youssef M. 1 Mohamed, Shakir 1 Nalisnick, Eric 1 Neumann, Gerhard 1 Oseledets, Ivan Valer’evich 1 Papamakarios, George 1 Pavez, Juan 1 Perdikaris, Paris G. 1 Ponomarev, E. 1 Rezende, Danilo Jimenez 1 Teichmann, Josef 1 Zabaras, Nicholas J. 1 Zech, Jakob 1 Zhong, Mingjun 1 Zhu, Yinhao all top 5 Cited in 6 Serials 2 Journal of Machine Learning Research (JMLR) 1 Journal of Computational Physics 1 Constructive Approximation 1 Computational Mathematics and Mathematical Physics 1 Proceedings of the National Academy of Sciences of the United States of America 1 SIAM Journal on Mathematics of Data Science all top 5 Cited in 10 Fields 5 Computer science (68-XX) 3 Numerical analysis (65-XX) 2 Statistics (62-XX) 1 Several complex variables and analytic spaces (32-XX) 1 Partial differential equations (35-XX) 1 Approximations and expansions (41-XX) 1 Calculus of variations and optimal control; optimization (49-XX) 1 Global analysis, analysis on manifolds (58-XX) 1 Fluid mechanics (76-XX) 1 Systems theory; control (93-XX) Citations by Year