GPyTorch swMATH ID: 35483 Software Authors: Jacob R. Gardner, Geoff Pleiss, David Bindel, Kilian Q. Weinberger, Andrew Gordon Wilson Description: GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration. Despite advances in scalable models, the inference tools used for Gaussian processes (GPs) have yet to fully capitalize on developments in computing hardware. We present an efficient and general approach to GP inference based on Blackbox Matrix-Matrix multiplication (BBMM). BBMM inference uses a modified batched version of the conjugate gradients algorithm to derive all terms for training and inference in a single call. BBMM reduces the asymptotic complexity of exact GP inference from O(n3) to O(n2). Adapting this algorithm to scalable approximations and complex GP models simply requires a routine for efficient matrix-matrix multiplication with the kernel and its derivative. In addition, BBMM uses a specialized preconditioner to substantially speed up convergence. In experiments we show that BBMM effectively uses GPU hardware to dramatically accelerate both exact GP inference and scalable approximations. Additionally, we provide GPyTorch, a software platform for scalable GP inference via BBMM, built on PyTorch. Homepage: https://gpytorch.ai Source Code: http://github.com/cornellius-gp/gpytorch Dependencies: PyTorch Keywords: Machine Learning; arXiv_cs.LG; arXiv_stat.ML; Gaussian processes; GPs; GPU Acceleration; Blackbox Matrix-Matrix multiplication; BBMM; PyTorch Related Software: PyTorch; TensorFlow; GPflow; SciPy; Python; Adam; PPINN; BoTorch; PILCO; JAX; UCI-ml; Scikit; U-Net; GitHub; GPML; GPy; PyTorch Lightning; ImageNet; MNIST; NumPy Cited in: 18 Publications Standard Articles 1 Publication describing the Software Year GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration Jacob R. Gardner, Geoff Pleiss, David Bindel, Kilian Q. Weinberger, Andrew Gordon Wilson 2018 all top 5 Cited by 68 Authors 2 Adams, Nikolaus A. 2 Ammar, Haitham Bou 2 Cortinovis, Alice 2 Cowen-Rivers, Alexander I. 2 Kressner, Daniel 2 Wang, Jun 2 Winter, Josef M. 1 Abdullah, Mohammed Amin 1 Adami, Stefan 1 Akhatov, Iskander Sh. 1 Borovitskiĭ, Vyacheslav A. 1 Bostanabad, Ramin 1 Burt, David R. 1 Carmel, Nadav 1 Chandramowlishwaran, Aparna 1 Charlier, Benjamin 1 Choshen, Era 1 Collin, François-David 1 Cuomo, Salvatore 1 Deisenroth, Marc Peter 1 Durif, Ghislain 1 Feng, Yiqi 1 Feydy, Jean 1 Frasconi, Paolo 1 Giampaolo, Fabio 1 Glaunès, Joan Alexis 1 Griffiths, Ryan-Rhys 1 Grosnit, Antoine 1 Herlands, William 1 Jäger, Manfred 1 Kaiser, J. W. J. 1 Kirby, Robert M. II 1 Klasnja, Predrag 1 Koutsourelakis, Phaedon-Stelios 1 Liao, Peng 1 Massei, Stefano 1 Moens, Vincent 1 Mostowsky, Peter 1 Murphy, Susan A. 1 Narayan, Akil C. 1 Neill, Daniel B. 1 Nickisch, Hannes 1 Or, Dvir Ben 1 Palenicek, Daniel 1 Penwarden, Michael 1 Perdikaris, Paris G. 1 Piccialli, Francesco 1 Planas, Robert 1 Raissi, Maziar 1 Rasmussen, Carl Edward 1 Rozza, Gianluigi 1 Schiano Di Cola, Vincenzo 1 Schranner, Felix S. 1 Shabat, Gil 1 Shi, Tianyi 1 Sootla, Aivar 1 Terenin, Alexander 1 Tibo, Alessandro 1 Tomkins, Sabina 1 Townsend, Alex 1 Tutunov, Rasul 1 van der Wilk, Mark 1 Wang, Hengjie 1 Wilson, Andrew Gordon 1 Wilson, James T. 1 Zabaras, Nicholas J. 1 Zhe, Shandian 1 Zhu, Yinhao all top 5 Cited in 7 Serials 6 Journal of Machine Learning Research (JMLR) 3 Journal of Computational Physics 3 SIAM Journal on Matrix Analysis and Applications 2 Computer Methods in Applied Mechanics and Engineering 2 Machine Learning 1 Journal of Scientific Computing 1 Foundations of Computational Mathematics all top 5 Cited in 10 Fields 12 Computer science (68-XX) 7 Numerical analysis (65-XX) 3 Fluid mechanics (76-XX) 3 Biology and other natural sciences (92-XX) 2 Partial differential equations (35-XX) 2 Probability theory and stochastic processes (60-XX) 1 Linear and multilinear algebra; matrix theory (15-XX) 1 Approximations and expansions (41-XX) 1 Statistics (62-XX) 1 Operations research, mathematical programming (90-XX) Citations by Year