×

ADADELTA

swMATH ID: 39429
Software Authors: Matthew D. Zeiler
Description: ADADELTA: An Adaptive Learning Rate Method. We present a novel per-dimension learning rate method for gradient descent called ADADELTA. The method dynamically adapts over time using only first order information and has minimal computational overhead beyond vanilla stochastic gradient descent. The method requires no manual tuning of a learning rate and appears robust to noisy gradient information, different model architecture choices, various data modalities and selection of hyperparameters. We show promising results compared to other methods on the MNIST digit classification task using a single machine and on a large scale voice dataset in a distributed cluster environment.
Homepage: https://paperswithcode.com/paper/adadelta-an-adaptive-learning-rate-method
Related Software: Adam; AdaGrad; RMSprop; ImageNet; AlexNet; TensorFlow; PRMLT; PyTorch; darch; L-BFGS; GitHub; MNIST; Keras; CIFAR; ADVI; ElemStatLearn; Saga; R; Tensor2Tensor; LaplacianSmoothing-GradientDescent
Cited in: 70 Publications
all top 5

Cited by 194 Authors

5 Nott, David John
3 Smith, Michael Stanley
3 Tran, Minh Ngoc
3 Yan, Liang
3 Zhou, Tao
2 Bilionis, Ilias
2 Bottou, Léon
2 Fedkiw, Ronald P.
2 Lee, Jaehong
2 Loaiza-Maya, Rubén
2 Ong, Victor M. H.
2 Osher, Stanley Joel
2 Roosta, Fred
2 Tripathy, Rohit K.
2 Wang, Bao
1 Abdessalem, Talel
1 Babenko, Mikhail
1 Bai, Jinshuai
1 Balu, Aditya
1 Bao, Michael
1 Baraniuk, Richard G.
1 Barekat, Farzin
1 Basir, Shamsulhaq
1 Belotto da Silva, André Ricardo
1 Bertozzi, Andrea Louise
1 Bifet, Albert
1 Bolmier, Geoffrey
1 Bouzerdoum, Abdesselam
1 Brandman, David M.
1 Burkhart, Michael C.
1 Chen, Jingrun
1 Chen, Tom
1 Chung, Julianne M.
1 Chung, Matthias
1 Clausen, Johan Bjerre Bach
1 Cortés-Mendoza, Jorge M.
1 Curtis, Frank E.
1 De Loera, Jesús A.
1 De, Subhayan
1 Dingreville, Rémi
1 Do, Dieu T. T.
1 Do, Xuan Anh
1 Doostan, Alireza
1 Drovandi, Christopher C.
1 Du, Qiang
1 Erway, Jennifer B.
1 Fang, Chih-Hao
1 Feng, Miria
1 Feng, Wenying
1 Forestier, Germain
1 Franco, Brian
1 Gao, Junbin
1 Gazeau, Maxime
1 Geete, Kanu
1 Geng, Zhenglin
1 Gerlach, Richard H.
1 Ghadai, Sambit
1 Gomes, Heitor Murilo
1 Göttlich, Simone
1 Grama, Ananth Y.
1 Griffin, Joshua D.
1 Gu, Yuantong
1 Guo, Jun
1 Ha, Jingwan
1 Haddock, Jamie
1 Halford, Max
1 Han, Xianpei
1 Harrison, Matthew T.
1 Hebishima, Hana
1 Henderson, Donna
1 Hochberg, Leigh R.
1 Huck, Nicolas
1 Hyde, David A. B.
1 Idoumghar, Lhassane
1 Inage, Sin-ichi
1 Ismail Fawaz, Hassan
1 Jeong, Hyogu
1 Jeong, Jong-Hyeon
1 Jia, Yichen
1 Kangal, Evrim Ersin
1 Kantas, Nikolas
1 Karniadakis, George Em
1 Karumuri, Sharmila
1 Kim, Minseok
1 Kim, Sehwan
1 Kim, Sundong
1 Knapp, Stephan
1 Kohn, Robert J.
1 Koike, Atsushi
1 Kojima, Ryosuke
1 Krauss, Christopher
1 Krishnamurthy, Adarsh
1 Kuchukov, Viktor
1 Kuchukova, N.
1 Kurihara, Masahito
1 Kylasa, Sudhir
1 Lai, Shaopeng
1 Lee, Jae-Gil
1 Lee, Seunghye
1 Levantesi, Susanna
...and 94 more Authors
all top 5

Cited in 45 Serials

6 Journal of Computational Physics
6 Journal of Computational and Graphical Statistics
4 Journal of Machine Learning Research (JMLR)
4 Communications in Computational Physics
3 Computer Methods in Applied Mechanics and Engineering
3 Neural Computation
2 Machine Learning
2 Journal of Statistical Computation and Simulation
2 Statistics and Computing
1 Inverse Problems
1 Physics Reports
1 Psychometrika
1 Mathematics and Computers in Simulation
1 Programming and Computer Software
1 Computer Aided Geometric Design
1 Journal of Computational Mathematics
1 Computers & Operations Research
1 Computational Mechanics
1 International Journal of Approximate Reasoning
1 Journal of Scientific Computing
1 Computational Statistics
1 Annals of Physics
1 Applied Mathematical Modelling
1 European Journal of Operational Research
1 SIAM Review
1 Stochastic Processes and their Applications
1 Computational Statistics and Data Analysis
1 SIAM Journal on Optimization
1 The Journal of Artificial Intelligence Research (JAIR)
1 Annals of Mathematics and Artificial Intelligence
1 Optimization Methods & Software
1 Data Mining and Knowledge Discovery
1 Interdisciplinary Information Sciences (IIS)
1 Probability in the Engineering and Informational Sciences
1 Scandinavian Actuarial Journal
1 Archives of Computational Methods in Engineering
1 International Journal of Computational Methods
1 SIAM Journal on Imaging Sciences
1 Journal of Control Science and Engineering
1 Mathematical Programming Computation
1 Journal of the Operations Research Society of China
1 SIAM/ASA Journal on Uncertainty Quantification
1 Research in the Mathematical Sciences
1 Annals of Applied Mathematics
1 Mathematical Foundations of Computing

Citations by Year