×

Saga

swMATH ID: 39677
Software Authors: Aaron Defazio, Francis Bach, Simon Lacoste-Julien
Description: SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives. In this work we introduce a new optimisation method called SAGA in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates. SAGA improves on the theory behind SAG and SVRG, with better theoretical convergence rates, and has support for composite objectives where a proximal operator is used on the regulariser. Unlike SDCA, SAGA supports non-strongly convex problems directly, and is adaptive to any inherent strong convexity of the problem. We give experimental results showing the effectiveness of our method.
Homepage: https://paperswithcode.com/paper/saga-a-fast-incremental-gradient-method-with
Related Software: AdaGrad; LIBSVM; Finito; Adam; SpiderBoost; ProxSARAH; HOGWILD; UNLocBoX; ElemStatLearn; RMSprop; CIFAR; MNIST; L-BFGS; Pegasos; ADADELTA; ImageNet; SGD-QN; Jellyfish; AlexNet; RCV1
Cited in: 166 Documents
all top 5

Cited by 372 Authors

12 Richtárik, Peter
5 Gürbüzbalaban, Mert
5 Nguyen, Lam M.
5 Schönlieb, Carola-Bibiane
5 Tran Dinh Quoc
4 Duchi, John C.
4 Gorbunov, Eduard
4 Lan, Guanghui
4 Mairal, Julien
4 Mokhtari, Aryan
4 Phan, Dzung T.
4 Takáč, Martin
4 Zhang, Tong
3 Bach, Francis R.
3 Davis, Damek Shea
3 Driggs, Derek
3 Ehrhardt, Matthias Joachim
3 Gasnikov, Aleksandr V.
3 Guo, Tiande
3 Liang, Jingwei
3 Lin, Qihang
3 Mishchenko, Konstantin
3 Parrilo, Pablo A.
3 Ribeiro, Alejandro R.
3 Wen, Zaiwen
3 Xiao, Xiantao
3 Yang, Tianbao
3 Zhou, Yi
2 Benning, Martin
2 Berahas, Albert S.
2 Beznosikov, Aleksandr
2 Carmon, Yair
2 Cevher, Volkan
2 Dai, Yu-Hong
2 Deng, Cheng
2 Gower, Robert Mansel
2 Gu, Bin
2 Gu, Quanquan
2 Han, Congying
2 Han, Deren
2 Harchaoui, Zaid
2 He, Lulu
2 Horváth, Samuel
2 Huang, Heng
2 Iutzeler, Franck
2 Jalilzadeh, Afrooz
2 Jordan, Michael Irwin
2 Kovalev, Dmitry P.
2 Lacoste-Julien, Simon
2 Latafat, Puya
2 Lei, Lihua
2 Li, Zhize
2 Lin, Hongzhou
2 Liu, Xinwei
2 Liu, Yan
2 Loizou, Nicolas
2 Luo, Zhijian
2 Malick, Jérôme
2 Milzarek, Andre
2 Moulines, Eric
2 Nedić, Angelia
2 Nguyen, Phuong Ha
2 Nocedal, Jorge
2 Osher, Stanley Joel
2 Palagi, Laura
2 Patrinos, Panagiotis
2 Pham, Nhan H.
2 Qian, Yuntao
2 Scheinberg, Katya
2 Sidford, Aaron
2 Sun, Jie
2 Taylor, Adrien B.
2 Themelis, Andreas
2 van Dijk, Marten
2 Vũ, Bằng Công
2 Wang, Qingsong
2 Wang, Xiao
2 Xiao, Lin
2 Xu, Pan
2 Ye, Jimin
2 Yu, Tengteng
2 Zhang, Jin
2 Zhang, Xiaoqun
2 Zhou, Dongruo
1 Agarwal, Naman
1 Ahookhosh, Masoud
1 Alacaoglu, Ahmet
1 An, Nguyen Thai
1 Aravkin, Aleksandr Y.
1 Arjevani, Yossi
1 Babanezhad, Reza
1 Barekat, Farzin
1 Barrio, Roberto
1 Başar, Tamer
1 Belkin, Mikhail
1 Belomestny, Denis
1 Betcke, Marta M.
1 Bian, Fengmiao
1 Bianchi, Pascal
1 Blanchet, Jose H.
...and 272 more Authors
all top 5

Cited in 44 Serials

24 SIAM Journal on Optimization
24 Journal of Machine Learning Research (JMLR)
16 Mathematical Programming. Series A. Series B
12 Computational Optimization and Applications
7 Optimization Methods & Software
6 Journal of Optimization Theory and Applications
6 Machine Learning
4 Mathematics of Operations Research
4 Optimization Letters
4 SIAM Journal on Mathematics of Data Science
3 Journal of Scientific Computing
3 Journal of Global Optimization
3 SIAM Journal on Imaging Sciences
3 Journal of the Operations Research Society of China
2 Inverse Problems
2 Information Sciences
2 Neural Computation
2 Numerical Algorithms
2 SIAM Journal on Scientific Computing
2 Analysis and Applications (Singapore)
2 International Journal of Wavelets, Multiresolution and Information Processing
2 Acta Numerica
2 Journal of Industrial and Management Optimization
2 Set-Valued and Variational Analysis
2 SIAM/ASA Journal on Uncertainty Quantification
2 Research in the Mathematical Sciences
1 Applicable Analysis
1 Journal of the Franklin Institute
1 Mathematics of Computation
1 The Annals of Statistics
1 Applied Mathematics and Optimization
1 Automatica
1 Journal of Computational and Applied Mathematics
1 Theoretical Computer Science
1 Journal of Computational Mathematics
1 Physica D
1 Statistical Science
1 SIAM Journal on Matrix Analysis and Applications
1 Neural Networks
1 SIAM Review
1 Discrete and Continuous Dynamical Systems. Series S
1 Statistics and Computing
1 Annals of Applied Mathematics
1 European Mathematical Society Magazine

Citations by Year