×

Benchmarking optimization software – a (Hi)story. (English) Zbl 1453.90003

Summary: For more than twenty years, we are benchmarking optimization software. In late 2018, an event had a major impact (the “story”) on this service which had gained considerable notoriety. The history up to this point is sketched and the developments to late 2019 and hopefully beyond are described.

MSC:

90-03 History of operations research and mathematical programming
90-04 Software, source code, etc. for problems pertaining to operations research and mathematical programming
01A61 History of mathematics in the 21st century
90C60 Abstract computational complexity for mathematical programming problems
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Benchmarks for Optimization Software, http://plato.asu.edu/bench.html
[2] Decision Tree for Optimization Software, http://plato.asu.edu/guide.html
[3] A Selection of Older Benchmarks, http://plato.asu.edu/ftp/older-benchmarks
[4] Talk at INFORMS 2006, http://plato.asu.edu/talks/pitt1.pdf
[5] NEOS Server for Optimization, http://neos-server.org
[6] Talk at INFORMS 2008, http://plato.asu.edu/talks/wash.pdf
[7] A Collection of Testproblems in PDE-Constrained Optimization, http://plato.asu.edu/pdecon.html
[8] MIPLIB2010, http://miplib2010.zib.de
[9] Talk at EURO 2010, http://plato.asu.edu/talks/euro2010-1.pdf
[10] Talk at INFORMS 2011, http://plato.asu.edu/talks/mittelmann-bench.pdf
[11] A Note on Performance Profiles for Benchmarking Software, http://www.numerical.rl.ac.uk/people/j_scott/publications/2016/GouldScott.2016_TOMS.pdf · Zbl 1369.65202
[12] MIPLIB2010 - Mixed Integer Programming Library version 5, http://mpc.zib.de/index.php/MPC/article/view/56/28
[13] Talk at EURO 2013, http://plato.asu.edu/talks/euro2013.pdf
[14] Talk at ISMP 2015, http://plato.asu.edu/talks/ismp2015.pdf
[15] Talk at INFORMS 2016, http://plato.asu.edu/talks/informs2016-bench.pdf
[16] Talk at INFORMS 2017, http://plato.asu.edu/talks/informs2017.pdf
[17] MIPCL solver, http://www.mipcl-cpp.appspot.com/index.html
[18] MIPLIB2017, http://www.optimization-online.org/DB_HTML/2019/07/7285.html
[19] MIPLIB2017, http://miplib.zib.de
[20] Talk at ISMP 2018, http://plato.asu.edu/talks/ismp2018.pdf
[21] Talk at INFORMS 2018, http://plato.asu.edu/talks/informs2018.pdf
[22] Announcement by Gurobi, http://plato.asu.edu/ftp/apology.pdf
[23] Session at INFORMS 2018, https://www.abstractsonline.com/pp8/#!/4701/session/566
[24] FICO Blog, https://community.fico.com/s/page/a5Q2E000000DtboUAC/fico1490
[25] Talk at EURO 2019, http://plato.asu.edu/talks/euro2019.pdf
[26] Talk at INFORMS 2019, http://plato.asu.edu/talks/informs2019.pdf
[27] CBLIB, http://cblib.zib.de
[28] QPLIB, http://qplib.zib.de · Zbl 1435.90099
[29] Solving Constraint Integer Programs, http://scip.zib.de · Zbl 1171.90476
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.