IQC-Game swMATH ID: 39462 Software Authors: Zhang, Guodong; Bao, Xuchan; Lessard, Laurent; Grosse, Roger Description: A unified analysis of first-order methods for smooth games via integral quadratic constraints. The theory of integral quadratic constraints (IQCs) allows the certification of exponential convergence of interconnected systems containing nonlinear or uncertain elements. In this work, we adapt the IQC theory to study first-order methods for smooth and strongly-monotone games and show how to design tailored quadratic constraints to get tight upper bounds of convergence rates. Using this framework, we recover the existing bound for the gradient method (GD), derive sharper bounds for the proximal point method (PPM) and optimistic gradient method (OG), and provide for the first time a global convergence rate for the negative momentum method (NM) with an iteration complexity (mathcal{O}(kappa^{1.5})), which matches its known lower bound. In addition, for time-varying systems, we prove that the gradient method with optimal step size achieves the fastest provable worst-case convergence rate with quadratic Lyapunov functions. Finally, we further extend our analysis to stochastic games and study the impact of multiplicative noise on different algorithms. We show that it is impossible for an algorithm with one step of memory to achieve acceleration if it only queries the gradient once per batch (in contrast with the stochastic strongly-convex optimization setting, where such acceleration has been demonstrated). However, we exhibit an algorithm which achieves acceleration with two gradient queries per batch. Our code is made public at url{https://github.com/gd-zhang/IQC-Game}. Homepage: https://jmlr.csail.mit.edu/papers/v22/20-1068.html Source Code: https://github.com/gd-zhang/IQC-Game Keywords: smooth game optimization; monotone variational inequality; first-order methods; integral quadratic constraints; dynamical systems Related Software: AlexNet; Wasserstein GAN; SBEED; GitHub; ImageNet; PESTO; CVXPY; Macaulay2 Cited in: 1 Publication Standard Articles 1 Publication describing the Software, including 1 Publication in zbMATH Year A unified analysis of first-order methods for smooth games via integral quadratic constraints. Zbl 07370620Zhang, Guodong; Bao, Xuchan; Lessard, Laurent; Grosse, Roger 2021 Cited by 4 Authors 1 Bao, Xuchan 1 Grosse, Roger 1 Lessard, Laurent 1 Zhang, Guodong Cited in 1 Serial 1 Journal of Machine Learning Research (JMLR) Cited in 1 Field 1 Computer science (68-XX) Citations by Year