×

Enhancing evolutionary fuzzy systems for multi-class problems: distance-based relative competence weighting with truncated confidences (DRCW-TC). (English) Zbl 1352.68201

Summary: Classification problems with multiple classes suppose a challenge in Data Mining tasks. There is a difficulty inherent to the learning process when trying to find the most adequate discrimination functions among the different concepts within the dataset. Using Fuzzy Rule Based Classification Systems in general, and Evolutionary Fuzzy Systems in particular, provide the advantage of describing smoother borderline areas, thanks to the linguistic label-based representation.
In multi-classification, the pairwise learning approach (One-vs-One) has gained a notorious attention. However, there is certain dependence between the goodness of the confidence degrees or scores of binary classifiers, and the final performance shown by the global model. Regarding this fact, the problem of non-competent classifiers is of special relevance. It occurs when a binary classifier outputs a positive score for a couple of classes unrelated with the input example, which may degrade the final accuracy. Precisely, the previously exposed properties of fuzzy classifiers make them more prone to the former condition. In this paper, we propose an extension of the distance-based combination strategy to overcome this non-competence problem. It is based on the truncation of the confidence degrees of the classes prior to the distance-based tuning. This allows taking advantage of the good classification abilities of Evolutionary Fuzzy Systems, while diminishing the adverse effect of the aforementioned non-competence. Experimental results, using FARC-HD with overlap functions as the fuzzy learning algorithm, show that this new adaptation of the Distance-based Relative Competence Weighting model outperforms both the OVO and standard distance-based approaches, and it is competitive with robust classifiers such as Support Vector Machines.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
62H30 Classification and discrimination; cluster analysis (statistical aspects)
62H86 Multivariate analysis and fuzziness

Software:

WEKA; KEEL; C4.5; JStatCom; LIBSVM
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Han, J.; Kamber, M.; Pei, J., Data Mining: Concepts and Techniques (2011), Morgan Kaufmann: Morgan Kaufmann San Mateo, CA, USA
[2] Lorena, A. C.; Carvalho, A. C.; Gama, J. M., A review on the combination of binary classifiers in multiclass problems, Artif. Intell. Rev., 30, 1-4, 19-37 (2008)
[3] Galar, M.; Fernández, A.; Barrenechea, E.; Bustince, H.; Herrera, F., A review on ensembles for class imbalance problem: bagging, boosting and hybrid based approaches, IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev., 42, 4, 463-484 (2012)
[4] Allwein, E. L.; Schapire, R. E.; Singer, Y., Reducing multiclass to binary: a unifying approach for margin classifiers, J. Mach. Learn. Res., 1, 113-141 (2000) · Zbl 1013.68175
[5] Dietterich, T. G.; Bakiri, G., Solving multiclass learning problems via error-correcting output codes, J. Artif. Intell. Res., 2, 263-286 (1995) · Zbl 0900.68358
[6] Knerr, S.; Personnaz, L.; Dreyfus, G., Single-layer learning revisited: a stepwise procedure for building and training a neural network, (Fogelman Soulié, F.; Hérault, J., Neurocomputing: Algorithms, Architectures and Applications. Neurocomputing: Algorithms, Architectures and Applications, NATO ASI Series, vol. F68 (1990), Springer-Verlag), 41-50
[7] Hastie, T.; Tibshirani, R., Classification by pairwise coupling, Ann. Stat., 26, 2, 451-471 (1998) · Zbl 0932.62071
[8] Fürnkranz, J., Round Robin classification, J. Mach. Learn. Res., 2, 721-747 (2002) · Zbl 1033.68086
[9] Galar, M.; Fernández, A.; Barrenechea, E.; Bustince, H.; Herrera, F., An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes, Pattern Recognit., 44, 8, 1761-1776 (2011)
[10] Galar, M.; Fernandez, A.; Barrenechea, E.; Herrera, F., Empowering difficult classes with a similarity-based aggregation in multi-class classification problems, Inf. Sci., 264, 135-157 (2014) · Zbl 1335.68201
[11] Kuncheva, L. I., Combining Pattern Classifiers: Methods and Algorithms (2004), Wiley-Interscience · Zbl 1066.68114
[12] Wozniak, M.; Graña, M.; Corchado, E., A survey of multiple classifier systems as hybrid systems, Inf. Fusion, 16, 3-17 (2014)
[13] Hüllermeier, E.; Vanderlooy, S., Combining predictions in pairwise classification: an optimal adaptive voting strategy and its relation to weighted voting, Pattern Recognit., 43, 1, 128-142 (2010) · Zbl 1191.68578
[14] Galar, M.; Fernandez, A.; Barrenechea, E.; Bustince, H.; Herrera, F., Dynamic classifier selection for one-vs-one strategy: avoiding non-competent classifiers, Pattern Recognit., 46, 12, 3412-3424 (2013)
[15] Galar, M.; Fernandez, A.; Barrenechea, E.; Herrera, F., DRCW-OVO: distance-based relative competence weighting combination for one-vs-one strategy in multi-class problems, Pattern Recognit., 48, 1, 28-42 (2015)
[16] Ishibuchi, H.; Nakashima, T.; Nii, M., Classification and Modeling with Linguistic Information Granules: Advanced Approaches to Linguistic Data Mining (2004), Springer-Verlag · Zbl 1060.68102
[17] Gacto, M.; Alcala, R.; Herrera, F., Interpretability of linguistic fuzzy rule-based systems: an overview of interpretability measures, Inf. Sci., 181, 20, 4340-4360 (2011)
[18] Fernandez, A.; Lopez, V.; del Jesus, M. J.; Herrera, F., Revisiting evolutionary fuzzy systems: taxonomy, applications, new trends and challenges, Knowl.-Based Syst., 80, 109-121 (2015)
[19] Alshomrani, S.; Bawakid, A.; Shim, S.-O.; Fernandez, A.; Herrera, F., A proposal for evolutionary fuzzy systems using feature weighting: dealing with overlapping in imbalanced datasets, Knowl.-Based Syst., 73, 1-17 (2015)
[20] Sanz, J. A.; Fernandez, A.; Bustince, H.; Herrera, F., IVTURS: a linguistic fuzzy rule-based classification system based on a new interval-valued fuzzy reasoning method with tuning and rule selection, IEEE Trans. Fuzzy Syst., 21, 3, 399-411 (2013)
[21] Alcalá-Fdez, J.; Fernández, A.; Luengo, J.; Derrac, J.; García, S.; Sánchez, L.; Herrera, F., KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework, J. Mult.-Valued Log. Soft Comput., 17, 2-3, 255-287 (2011)
[22] Demšar, J., Statistical comparisons of classifiers over multiple data sets, J. Mach. Learn. Res., 7, 1-30 (2006) · Zbl 1222.68184
[23] Garcia, S.; Herrera, F., An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons, J. Mach. Learn. Res., 9, 2677-2694 (2008) · Zbl 1225.68178
[24] García, S.; Fernández, A.; Luengo, J.; Herrera, F., Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power, Inf. Sci., 180, 10, 2044-2064 (2010)
[25] Ishibuchi, H.; Nozaki, K.; Tanaka, H., Distributed representation of fuzzy rules and its application to pattern classification, Fuzzy Sets Syst., 52, 1, 21-32 (1992)
[26] Cordon, O.; del Jesus, M. J.; Herrera, F., A proposal on reasoning methods in fuzzy rule-based classification systems, Int. J. Approx. Reason., 20, 1, 21-45 (1999)
[27] Ishibuchi, H.; Yamamoto, T., Rule weight specification in fuzzy rule-based classification systems, IEEE Trans. Fuzzy Syst., 13, 428-435 (2005)
[28] Elkano, M.; Galar, M.; Sanz, J.; Fernandez, A.; Barrenechea, E.; Herrera, F.; Bustince, H., Enhancing multi-class classification in FARC-HD fuzzy classifier: on the synergy between \(n\)-dimensional overlap functions and decomposition strategies, IEEE Trans. Fuzzy Syst., 23, 5, 1562-1580 (2015)
[29] Cordón, O.; Gomide, F.; Herrera, F.; Hoffmann, F.; Magdalena, L., Ten years of genetic fuzzy systems: current framework and new trends, Fuzzy Sets Syst., 141, 1, 5-31 (2004) · Zbl 1050.93513
[30] Cordon, O.; Herrera, F.; Hoffmann, F.; Magdalena, L., Genetic Fuzzy Systems. Evolutionary Tuning and Learning of Fuzzy Knowledge Bases (2001), World Scientific: World Scientific Singapore, Republic of Singapore · Zbl 1042.68098
[31] Cordon, O., A historical review of evolutionary learning methods for mamdani-type fuzzy rule-based systems: designing interpretable genetic fuzzy systems, Int. J. Approx. Reason., 52, 6, 894-913 (2011)
[32] Herrera, F., Genetic fuzzy systems: taxonomy, current research trends and prospects, Evol. Intel., 1, 27-46 (2008)
[33] Alcalá-Fdez, J.; Alcalá, R.; Herrera, F., A fuzzy association rule-based classification model for high-dimensional problems with genetic rule selection and lateral tuning, IEEE Trans. Fuzzy Syst., 19, 5, 857-872 (2011)
[34] Han, J.; Kamber, M., Data Mining. Concepts and Techniques (2006), Morgan Kaufmann · Zbl 1445.68004
[35] Zhang, C.; Zhang, S., Association Rule Mining, Models and Algorithms, Lecture Notes in Computer Science, vol. 2307 (2002), Springer · Zbl 0998.68047
[36] Kavsek, B.; Lavrac, N., Apriori-sd: adapting association rule learning to subgroup discovery, Appl. Artif. Intell., 20, 7, 543-583 (2006)
[37] Casillas, J.; Cordon, O.; del Jesus, M. J.; Herrera, F., Genetic tuning of fuzzy rule deep structures preserving interpretability and its interaction with fuzzy rule set reduction, IEEE Trans. Fuzzy Syst., 13, 1, 13-29 (2005)
[38] Alcala, R.; Alcala-Fdez, J.; Herrera, F., A proposal for the genetic lateral tuning of linguistic fuzzy systems and its interaction with rule selection, IEEE Trans. Fuzzy Syst., 15, 4, 616-635 (2007) · Zbl 1147.68063
[39] Bustince, H.; Fernandez, J.; Mesiar, R.; Montero, J.; Orduna, R., Overlap functions, Nonlinear Anal.: Theory, Methods Appl., 72, 3-4, 1488-1499 (2010) · Zbl 1182.26076
[40] Paternain, D.; Fernández, J.; Sola, H. B.; Mesiar, R.; Beliakov, G., Construction of image reduction operators using averaging aggregation functions, Fuzzy Sets Syst., 261, 87-111 (2015) · Zbl 1360.68881
[41] Lorena, A. C.; Carvalho, A. C.; Gama, J. M., A review on the combination of binary classifiers in multiclass problems, Artif. Intell. Rev., 30, 1-4, 19-37 (2008)
[42] Vapnik, V., Statistical Learning Theory (1998), Wiley: Wiley New York · Zbl 0935.62007
[43] Hsu, C. W.; Lin, C. J., A comparison of methods for multiclass support vector machines, IEEE Trans. Neural Netw., 13, 2, 415-425 (2002)
[44] Rifkin, R.; Klautau, A., In defense of one-vs-all classification, J. Mach. Learn. Res., 5, 101-141 (2004) · Zbl 1222.68287
[45] Alcalá-Fdez, J.; Sánchez, L.; García, S.; del Jesus, M. J.; Ventura, S.; Garrell, J. M.; Otero, J.; Romero, C.; Bacardit, J.; Rivas, V. M.; Fernández, J. C.; Herrera, F., KEEL: a software tool to assess evolutionary algorithms for data mining problems, Soft Comput., 13, 307-318 (2009)
[46] Hall, M.; Frank, E.; Holmes, G.; Pfahringer, B.; Reutemann, P.; Witten, I. H., The weka data mining software: an update, SIGKDD Explor. Newsl., 11, 10-18 (2009)
[47] Chang, C.-C.; Lin, C.-J., LIBSVM: a library for support vector machines, ACM Trans. Intell. Syst. Technol., 2, 27 (2011), software available at
[48] Fürnkranz, J., Round Robin classification, J. Mach. Learn. Res., 2, 721-747 (2002) · Zbl 1033.68086
[49] Sáez, J. A.; Galar, M.; Luengo, J.; Herrera, F., Analyzing the presence of noise in multi-class problems: alleviating its influence with the one-vs-one decomposition, Knowl. Inf. Syst., 38, 1, 179-206 (2014)
[50] Fernandez, A.; Calderon, M.; Barrenechea, E.; Bustince, H.; Herrera, F., Solving multi-class problems with linguistic fuzzy rule based classification systems based on pairwise learning and preference relations, Fuzzy Sets Syst., 161, 23, 3064-3080 (2010) · Zbl 1209.68412
[51] Elhag, S.; Fernandez, A.; Bawakid, A.; Alshomrani, S.; Herrera, F., On the combination of genetic fuzzy systems and pairwise learning for improving detection rates on intrusion detection systems, Expert Syst. Appl., 42, 1, 193-202 (2015)
[52] Hüllermeier, E.; Vanderlooy, S., Combining predictions in pairwise classification: an optimal adaptive voting strategy and its relation to weighted voting, Pattern Recognit., 43, 1, 128-142 (2010) · Zbl 1191.68578
[53] Galar, M.; Fernández, A.; Barrenechea, E.; Bustince, H.; Herrera, F., An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes, Pattern Recognit., 44, 8, 1761-1776 (2011)
[54] Galar, M.; Fernández, A.; Barrenechea, E.; Bustince, H.; Herrera, F., Dynamic classifier selection for one-vs-one strategy: avoiding non-competent classifiers, Pattern Recognit., 46, 12, 3412-3424 (2013)
[55] Friedman, J. H., Another approach to polychotomous classification (1996), Department of Statistics, Stanford University, Tech. rep.
[56] Wu, T. F.; Lin, C. J.; Weng, R. C., Probability estimates for multi-class classification by pairwise coupling, J. Mach. Learn. Res., 5, 975-1005 (2004) · Zbl 1222.68336
[57] Moreno-Torres, J. G.; Sáez, J. A.; Herrera, F., Study on the impact of partition-induced dataset shift on k-fold cross-validation, IEEE Trans. Neural Netw. Learn. Syst., 23, 8, 1304-1313 (2012)
[58] Moreno-Torres, J. G.; Raeder, T.; Aláiz-Rodríguez, R.; Chawla, N. V.; Herrera, F., A unifying view on dataset shift in classification, Pattern Recognit., 45, 1, 521-530 (2012)
[59] Candela, J. Q.; Sugiyama, M.; Schwaighofer, A.; Lawrence, N. D., Dataset Shift in Machine Learning (2009), The MIT Press
[60] Lopez, V.; Fernandez, A.; Herrera, F., On the importance of the validation technique for classification with imbalanced datasets: addressing covariate shift when data is skewed, Inf. Sci., 257, 1-13 (2014)
[61] Quinlan, J. R., C4.5: Programs for Machine Learning (1993), Morgan Kaufmann Publishers: Morgan Kaufmann Publishers San Mateo-California
[62] Platt, J. C., Fast Training of Support Vector Machines Using Sequential Minimal Optimization (1999), MIT Press: MIT Press Cambridge, MA, USA
[63] Platt, J. C., Probabilistic outputs for support vector machines and comparison to regularized likelihood methods, (Smola, A.; Bartlett, P.; Schölkopf, B.; Schuurmans, D., Advances in Large Margin Classifiers. Advances in Large Margin Classifiers, Cambridge, MA (2000))
[64] Wilson, D.; Martinez, T., Improved heterogeneous distance functions, J. Artif. Intell. Res., 6, 1-34 (1997) · Zbl 0894.68118
[65] García, S.; Herrera, F., An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons, J. Mach. Learn. Res., 9, 2607-2624 (2008)
[66] Holm, S., A simple sequentially rejective multiple test procedure, Scand. J. Stat., 6, 65-70 (1979) · Zbl 0402.62058
[67] Wilcoxon, F., Individual comparisons by ranking methods, Biom. Bull., 1, 6, 80-83 (1945)
[68] Zar, J. H., Biostatistical Analysis (1999), Prentice Hall: Prentice Hall Upper Saddle River, New Jersey
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.