zbMATH — the first resource for mathematics

An extension on “Statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. (English) Zbl 1225.68178
Summary: In a recently published paper [ibid. 7, 1–30 (2006; Zbl 1222.68184)], J. Demšar recommends a set of non-parametric statistical tests and procedures which can be safely used for comparing the performance of classifiers over multiple data sets. After studying the paper, we realize that the paper correctly introduces the basic procedures and some of the most advanced ones when comparing a control method. However, it does not deal with some advanced topics in depth. Regarding these topics, we focus on more powerful proposals of statistical procedures for comparing \(n \times n\) classifiers. Moreover, we illustrate an easy way of obtaining adjusted and comparable \(p\)-values in multiple comparison procedures.

68T05 Learning and adaptive systems in artificial intelligence
62G30 Order statistics; empirical distribution functions
Full Text: Link