ForwardDiff swMATH ID: 16106 Software Authors: Jarrett Revels, Miles Lubin, Theodore Papamarkou Description: Forward-Mode Automatic Differentiation in Julia. We present ForwardDiff, a Julia package for forward-mode automatic differentiation (AD) featuring performance competitive with low-level languages like C++. Unlike recently developed AD tools in other popular high-level languages such as Python and MATLAB, ForwardDiff takes advantage of just-in-time (JIT) compilation to transparently recompile AD-unaware user code, enabling efficient support for higher-order differentiation and differentiation using custom number types (including complex numbers). For gradient and Jacobian calculations, ForwardDiff provides a variant of vector-forward mode that avoids expensive heap allocation and makes better use of memory bandwidth than traditional vector mode. In our numerical experiments, we demonstrate that for nontrivially large dimensions, ForwardDiff’s gradient computations can be faster than a reverse-mode implementation from the Python-based autograd package. We also illustrate how ForwardDiff is used effectively within JuMP, a modeling language for optimization. According to our usage statistics, 41 unique repositories on GitHub depend on ForwardDiff, with users from diverse fields such as astronomy, optimization, finite element analysis, and statistics. This document is an extended abstract that has been accepted for presentation at the AD2016 7th International Conference on Algorithmic Differentiation. Homepage: https://github.com/JuliaDiff/ForwardDiff.jl Source Code: https://github.com/JuliaDiff/ForwardDiff.jl Dependencies: Julia Keywords: automatic differentiation; forward-mode AD; Julia package; JuMP modeling language Related Software: Julia; PyTorch; TensorFlow; Ipopt; JuMP; Stan; DiffSharp; Python; Matlab; DifferentialEquations.jl; ADiGator; AMPL; Bridge.jl; ddfermi; SciPy; LBFGS-B; GAMS; Cassette.jl; Pyomo; Gurobi Cited in: 20 Publications Standard Articles 1 Publication describing the Software Year Forward-Mode Automatic Differentiation in Julia Jarrett Revels, Miles Lubin, Theodore Papamarkou 2016 all top 5 Cited by 51 Authors 2 Fuhrmann, Jürgen 2 Gaudeul, Benoît 2 Orban, Dominique 2 Ranocha, Hendrik 1 Akbayrak, Semih 1 Ancellin, Matthieu 1 Aravkin, Aleksandr Y. 1 Arnaudon, Alexis 1 Baraldi, Robert 1 Baydin, Atılım Güneş 1 Burghardt, Jeff 1 Cancès, Clément 1 Cellier, Nicolas 1 Chainais-Hillairet, Claire 1 Chan, Jesse 1 Chhay, Marx 1 Darve, Eric 1 De Vries, Bert J. M. 1 Després, Bruno 1 Dunning, Iain 1 Fanaskov, Vladimir 1 Ford, Eric B. 1 Gassner, Gregor J. 1 Huchette, Joey 1 Leok, Melvin 1 Lindo, Alexey 1 Lubin, Miles 1 Milz, Johannes 1 Mukhopadhyay, Sanghasri 1 Oseledets, Ivan V. 1 Papamarkou, Theodore 1 Pearlmutter, Barak A. 1 R, Usha 1 Radul, Alexey Andreyevich 1 Ruyer-Quil, Christian 1 Sarı, Alp 1 Schauer, Moritz 1 Schmitt, Jeremy M. 1 Şenöz, İsmail 1 Shingel, Tatiana 1 Siqueira, Abel Soares 1 Siskind, Jeffrey Mark 1 Sommer, Stefan 1 Stuber, Matthew D. 1 Tartakovsky, Alexandre M. 1 Taylor, Christina G. 1 Ulbrich, Michael 1 van der Meulen, Frank Henri 1 Wilhelm, M. E. 1 Wormell, Caroline L. 1 Xu, Kailai all top 5 Cited in 16 Serials 2 Journal of Computational Physics 2 Numerische Mathematik 2 SIAM Journal on Optimization 1 Computer Methods in Applied Mechanics and Engineering 1 Journal of Fluid Mechanics 1 BIT 1 Journal of Computational and Applied Mathematics 1 International Journal of Approximate Reasoning 1 SIAM Review 1 Computational Optimization and Applications 1 Optimization Methods & Software 1 Journal of Machine Learning Research (JMLR) 1 Comptes Rendus. Mathématique. Académie des Sciences, Paris 1 SIAM Journal on Imaging Sciences 1 Communications on Applied Mathematics and Computation 1 Foundations of Data Science all top 5 Cited in 12 Fields 14 Numerical analysis (65-XX) 5 Operations research, mathematical programming (90-XX) 3 Partial differential equations (35-XX) 3 Computer science (68-XX) 3 Fluid mechanics (76-XX) 2 Dynamical systems and ergodic theory (37-XX) 2 Probability theory and stochastic processes (60-XX) 2 Statistics (62-XX) 2 Optics, electromagnetic theory (78-XX) 1 Calculus of variations and optimal control; optimization (49-XX) 1 Mechanics of particles and systems (70-XX) 1 Biology and other natural sciences (92-XX) Citations by Year