OptNet swMATH ID: 42660 Software Authors: Brandon Amos, J. Zico Kolter Description: OptNet: Differentiable Optimization as a Layer in Neural Networks. This paper presents OptNet, a network architecture that integrates optimization problems (here, specifically in the form of quadratic programs) as individual layers in larger end-to-end trainable deep networks. These layers encode constraints and complex dependencies between the hidden states that traditional convolutional and fully-connected layers often cannot capture. We explore the foundations for such an architecture: we show how techniques from sensitivity analysis, bilevel optimization, and implicit differentiation can be used to exactly differentiate through these layers and with respect to layer parameters; we develop a highly efficient solver for these layers that exploits fast GPU-based batch solves within a primal-dual interior point method, and which provides backpropagation gradients with virtually no additional cost on top of the solve; and we highlight the application of these approaches in several problems. In one notable example, the method is learns to play mini-Sudoku (4x4) given just input and output games, with no a-priori information about the rules of the game; this highlights the ability of OptNet to learn hard constraints better than other neural architectures. Homepage: https://arxiv.org/abs/1703.00443 Source Code: https://github.com/locuslab/optnet Dependencies: Python Related Software: Adam; PyTorch; CVXGEN; Julia; DiffSharp; torchdiffeq; TensorFlow; RMSprop; AdaGrad; ImageNet; L-BFGS; ALTRO; Dojo; DiffPills; HYPLAS; BigGAN; Wasserstein GAN; Flow++; POT; pix2pix Cited in: 7 Publications all top 5 Cited by 25 Authors 1 Achterhold, Jan 1 Akrour, Riad 1 Askari, Armin 1 Atamna, Asma 1 Baydin, Atılım Güneş 1 Bertocchi, Carla 1 Chouzenoux, Emilie 1 Corbineau, Marie-Caroline 1 Darve, Eric 1 Fedkiw, Ronald P. 1 Geng, Zhenglin 1 Gu, Fangda 1 Kandukuri, Rama Krishna 1 Möller, Michael 1 Pearlmutter, Barak A. 1 Pesquet, Jean-Christophe 1 Peters, Jan 1 Prato, Marco 1 Radul, Alexey Andreyevich 1 Siskind, Jeffrey Mark 1 Stueckler, Joerg 1 Travacca, Bertrand 1 Tsai, Alicia 1 Xu, Kailai 1 Zhu, Weiqiang all top 5 Cited in 7 Serials 1 Inverse Problems 1 Journal of Computational Physics 1 Machine Learning 1 International Journal of Computer Vision 1 Journal of Machine Learning Research (JMLR) 1 Research in the Mathematical Sciences 1 SIAM Journal on Mathematics of Data Science all top 5 Cited in 9 Fields 4 Numerical analysis (65-XX) 4 Computer science (68-XX) 1 General and overarching topics; collections (00-XX) 1 Real functions (26-XX) 1 Partial differential equations (35-XX) 1 Probability theory and stochastic processes (60-XX) 1 Statistics (62-XX) 1 Operations research, mathematical programming (90-XX) 1 Information and communication theory, circuits (94-XX) Citations by Year