Tunability swMATH ID: 42100 Software Authors: Probst, Philipp; Boulesteix, Anne-Laure; Bischl, Bernd Description: Tunability: importance of hyperparameters of machine learning algorithms. Modern supervised machine learning algorithms involve hyperparameters that have to be set before running them. Options for setting hyperparameters are default values from the software package, manual configuration by the user or configuring them for optimal predictive performance by a tuning procedure. The goal of this paper is two-fold. Firstly, we formalize the problem of tuning from a statistical point of view, define data-based defaults and suggest general measures quantifying the tunability of hyperparameters of algorithms. Secondly, we conduct a large-scale benchmarking study based on 38 datasets from the OpenML platform and six common machine learning algorithms. We apply our measures to assess the tunability of their parameters. Our results yield default values for hyperparameters and enable users to decide whether it is worth conducting a possibly time consuming tuning strategy, to focus on the most important hyperparameters and to choose adequate hyperparameter spaces for tuning. Homepage: https://arxiv.org/abs/1802.09596 Source Code: https://github.com/PhilippPro/tunability Keywords: machine learning; supervised learning; classification; hyperparameters; tuning; meta-learning Related Software: OpenML; ranger; obliqueRF; XGBoost; caret; Hyperband; auto-sklearn; SMAC; Auto-WEKA; Hyperopt; Spearmint; AlexNet; BOHB; ImageNet; SNPInterForest; SHAFF; diversityForest; R; randomForestSRC; randomForest Cited in: 9 Publications Standard Articles 1 Publication describing the Software, including 1 Publication in zbMATH Year Tunability: importance of hyperparameters of machine learning algorithms. Zbl 1485.68226Probst, Philipp; Boulesteix, Anne-Laure; Bischl, Bernd 2019 all top 5 Cited by 27 Authors 2 Boulesteix, Anne-Laure Isabeau 2 Hornung, Roman 1 Barrieu, Pauline M. 1 Bischl, Bernd 1 Browne, James C. 1 Burns, Randal 1 Chen, YiNing 1 Chung, Jaewon 1 Falk, Benjamin 1 Huber, Marco F. 1 Maggioni, Mauro 1 Makariou, Despoina 1 Patsolic, Jesse L. 1 Priebe, Carey E. 1 Probst, Philipp 1 Royset, Johannes O. 1 Shen, Cencheng 1 Sheng, Xiang-Rong 1 Tomita, Tyler M. 1 Vogelstein, Joshua T. 1 Wets, Roger Jean-Baptiste 1 Yang, Zebin 1 Ye, Hanjia 1 Yim, Jason 1 Zhan, Dechuan 1 Zhang, Aijun 1 Zöller, Marc-André all top 5 Cited in 7 Serials 3 Journal of Machine Learning Research (JMLR) 1 The Annals of Statistics 1 Insurance Mathematics & Economics 1 Journal of Classification 1 Machine Learning 1 Computational Statistics and Data Analysis 1 The Journal of Artificial Intelligence Research (JAIR) Cited in 3 Fields 5 Computer science (68-XX) 4 Statistics (62-XX) 1 Game theory, economics, finance, and other social and behavioral sciences (91-XX) Citations by Year