×

A new nonlinear classifier with a penalized signed fuzzy measure using effective genetic algorithm. (English) Zbl 1192.68570

Summary: This paper proposes a new nonlinear classifier based on a generalized Choquet integral with signed fuzzy measures to enhance the classification accuracy and power by capturing all possible interactions among two or more attributes. This generalized approach was developed to address unsolved Choquet-integral classification issues such as allowing for flexible location of projection lines in \(n\)-dimensional space, automatic search for the least misclassification rate based on Choquet distance, and penalty on misclassified points. A special genetic algorithm is designed to implement this classification optimization with fast convergence. Both the numerical experiment and empirical case studies show that this generalized approach improves and extends the functionality of this Choquet nonlinear classification in more real-world multi-class multi-dimensional situations.

MSC:

68T10 Pattern recognition, speech recognition
68W05 Nonnumerical algorithms
28E10 Fuzzy measure theory
74P99 Optimization problems in solid mechanics

Software:

C4.5; SAS/STAT; SAS; UCI-ml
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[1] Quinlan, J. R., C4.5: Programs for Machine Learning (1993), Morgan Kaufmann: Morgan Kaufmann San Mateo, CA
[2] Breiman, L.; Friedman, J.; Olshen, R.; Stone, C., Classification and Regression Trees (1984), Chapman & Hall: Chapman & Hall New York · Zbl 0541.62042
[3] Friedman, J., Multivariate adaptive regression splines (with discussion), Annals of Statistics, 19, 1-141 (1982)
[4] Yuan, Y.; Shaw, M. J., Induction of fuzzy decision trees, Fuzzy Sets and Systems, 69, 125-139 (1995)
[5] Kohonen, T., Self-Organizing Maps (1995), Springer: Springer Heidelberg, Germany
[6] W.S. Sarle, Neural networks and statistical models, Presented at 19th Annual SAS Users Groups International Conference, Cary, NC, 1994.; W.S. Sarle, Neural networks and statistical models, Presented at 19th Annual SAS Users Groups International Conference, Cary, NC, 1994.
[7] Spooner, T. J.; Ordonez, R.; Maggiore, M.; Passino, K. M., Stable Adaptive Control and Estimation for Nonlinear Systems: Neural and Fuzzy Approximator Techniques (2002), Wiley: Wiley NY
[8] Grabisch, M., The representation of importance and interaction of features by fuzzy measures, Pattern Recognition Letters, 17, 567-575 (1996)
[9] Grabisch, M.; Nicolas, J. M., Classification by fuzzy integral: performance and tests, Fuzzy Sets and Systems, 65, 255-271 (1994)
[10] J.M. Keller, B. Yan, Possibility expression and its decision making algorithm, Presented at 1st IEEE International Conference on Fuzzy Systems, San Diego, CA, 1992.; J.M. Keller, B. Yan, Possibility expression and its decision making algorithm, Presented at 1st IEEE International Conference on Fuzzy Systems, San Diego, CA, 1992.
[11] Mikenina, L.; Zimmermann, H. J., Improved feature selection and classification by the 2-additive fuzzy measure, Fuzzy Sets and Systems, 107, 197-218 (1999) · Zbl 0958.68147
[12] Xu, K.; Wang, Z.; Heng, P. A.; Leung, K. S., Classification by nonlinear integral projections, IEEE Transactions on Fuzzy Systems, 11, 187-201 (2003)
[13] M. Liu, Z. Wang, Using generalized Choquet integral projections, Presented at IFSA 2005, 2005.; M. Liu, Z. Wang, Using generalized Choquet integral projections, Presented at IFSA 2005, 2005.
[14] Z. Wang, A new model of nonlinear multiregression by projection pursuit based on generalized Choquet integrals, Presented at FUZZ-IEEE2002, Hawaii, 2002.; Z. Wang, A new model of nonlinear multiregression by projection pursuit based on generalized Choquet integrals, Presented at FUZZ-IEEE2002, Hawaii, 2002.
[15] H. Wang, H. Sharif, Z. Wang, A new classifier based on genetic algorithm, Presented at IPMU 2006, 2006.; H. Wang, H. Sharif, Z. Wang, A new classifier based on genetic algorithm, Presented at IPMU 2006, 2006.
[16] Leung, K.; Wong, M.; Lam, W.; Wang, Z.; Xu, K., Learning nonlinear multiregression networks based on evolutionary computation, IEEE Transactions on Systems, Man, and Cybernetics, 32, 630-643 (2002)
[17] K. Xu, Z. Wang, P. Heng, K. Leung, Using a new type of nonlinear integral for multi-regression: an application of evolutionary algorithms in data mining, Presented at IEEE SMC’98, 1999.; K. Xu, Z. Wang, P. Heng, K. Leung, Using a new type of nonlinear integral for multi-regression: an application of evolutionary algorithms in data mining, Presented at IEEE SMC’98, 1999.
[18] Wang, Z.; Wong, M.; Fang, J.; Xu, K., Nonlinear nonnegative multiregressions based on Choquet integrals, International Journal of Approximate Reasoning, 25, 71-87 (2000) · Zbl 0968.68154
[19] Choquet, G., Theory of capacities, Annales de l’Institut Fourier, 5 (1953)
[20] Wang, Z.; Klir, G. J., Fuzzy Measure Theory (1992), Plenum Press: Plenum Press New York · Zbl 0812.28010
[21] Wang, Z.; Klir, G. J.; Wang, W., Monotone set functions defined by Choquet integral, Fuzzy Sets and Systems, 81, 241-250 (1996) · Zbl 0878.28011
[22] Wang, Z.; Klir, G. J., Choquet integrals and natural extensions of lower probabilities, International Journal of Approximate Reasoning, 16, 137-147 (1997) · Zbl 0935.28010
[23] Wang, Z., Convergence theorems for sequences of Choquet integrals, International Journal of General Systems, 26, 133-143 (1997) · Zbl 0880.28015
[24] Wang, Z.; Leung, K.; Klir, G. J., Integration on finite sets, International Journal of Intelligence Systems, 21, 1073-1092 (2006) · Zbl 1114.28010
[25] Wang, Z.; Yang, R.; Heng, P.; Leung, K., Real-valued Choquet integrals with fuzzy-valued integrand, Fuzzy Sets and Systems, 157, 256-269 (2006) · Zbl 1080.28011
[26] Wang, Z.; Xu, K.; Heng, P.; Leung, K., Indeterminate integrals with respect to nonadditive measures, Fuzzy Sets and Systems, 138, 485-495 (2003) · Zbl 1075.28009
[27] Wang, Z.; Xu, K.; Wang, J.; Klir, G., Using genetic algorithms to determine nonnegative monotone set functions for information fusion in environments with random perturbation, International Journal of Intelligent Systems, 14, 949-962 (1999) · Zbl 0937.68100
[28] Wang, W.; Wang, Z.; Klir, G. J., Genetic algorithms for determining fuzzy measures from data, Journal of Intelligent & Fuzzy Systems, 6, 171-183 (1998)
[29] Goldberg, E., Genetic Algorithms in Search, Optimization, and Machine Learning (1989), Addison-Wesley: Addison-Wesley Reading, MA · Zbl 0721.68056
[30] Mitchell, M.; Melanie, X., Introduction to Genetic Algorithms (1996), MIT Press: MIT Press Cambridge, MA
[31] Denneberg, D., Non-Additive Measure and Integral (1994), Kluwer Academic Publishers: Kluwer Academic Publishers Dordrecht · Zbl 0826.28002
[32] A. Asuncion, D.J. Newman, UCI Machine Learning Repository \(\langle\) http://www.ics.uci.edu/\( \sim\rangle \); A. Asuncion, D.J. Newman, UCI Machine Learning Repository \(\langle\) http://www.ics.uci.edu/\( \sim\rangle \)
[33] G.H. John, P. Langley, Estimating continuous distributions in Bayesian classifiers, Presented at 11th Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, San Mateo, 1995.; G.H. John, P. Langley, Estimating continuous distributions in Bayesian classifiers, Presented at 11th Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, San Mateo, 1995.
[34] R. Kohavi, Scaling up the accuracy of naive-Bayes classifiers: a decision tree hybrid, Presented at Second International Conference on Knowledge Discovery and Data Mining.; R. Kohavi, Scaling up the accuracy of naive-Bayes classifiers: a decision tree hybrid, Presented at Second International Conference on Knowledge Discovery and Data Mining.
[35] Frank, E.; Wang, Y.; Inglis, S.; Holmes, G.; Witten, I. H., Using model trees for classification, Machine Learning, 32, 63-76 (1998) · Zbl 0901.68167
[36] Keerthi, S. S.; Shevade, S. K.; Bhattacharyya, C.; Murthy, K. R.K., Improvements to Platt’s SMO algorithm for SVM classifier design, Neural Computation, 13, 637-649 (2001) · Zbl 1085.68629
[37] Kaburlasos, V. G.; Athanasiadis, I. N.; Mitkas,, P. A.; Petridis, V., Fuzzy lattice reasoning (FLR) classifier and its application on improved estimation of ambient ozone concentration, International, Journal of Approximate Reasoning, 45, 152-188 (2007) · Zbl 1123.68106
[38] Yuan, Y.; Shaw, M. J., Induction of fuzzy decision trees, Fuzzy Sets and Systems, 69, 2, 125-139 (1995)
[39] Fuzzy EBY software \(\langle\) http://www.eecs.berkeley.edu/\( \sim\rangle \); Fuzzy EBY software \(\langle\) http://www.eecs.berkeley.edu/\( \sim\rangle \)
[40] Y. Oyang, S.C. Hwang, Y. Qu, C. Chen, Z.W. Chen, Data classification with radial basis function networks based on a novel kernel density estimation algorithm, IEEE Transactions of Neural Networks, 16(1), 225-36.; Y. Oyang, S.C. Hwang, Y. Qu, C. Chen, Z.W. Chen, Data classification with radial basis function networks based on a novel kernel density estimation algorithm, IEEE Transactions of Neural Networks, 16(1), 225-36.
[41] \( \langle\) http://www.cs.waikato.ac.nz/ml/weka/\( \rangle \); \( \langle\) http://www.cs.waikato.ac.nz/ml/weka/\( \rangle \)
[42] \( \langle\) http://www.cs.umsl.edu/\( \sim\rangle \); \( \langle\) http://www.cs.umsl.edu/\( \sim\rangle \)
[43] SAS Institute Inc. SAS/STAT User’s Guide, version 9.1, SAS Institute Inc., Cary, NC, 2003.; SAS Institute Inc. SAS/STAT User’s Guide, version 9.1, SAS Institute Inc., Cary, NC, 2003.
[44] Temko, A.; Macho, D.; Nadeu, C., Fuzzy integral based information fusion for classification of highly confusable non-speech sounds, Pattern Recognition, 41, 1814-1823 (2008) · Zbl 1140.68482
[45] Strauss, O.; Comby, F., Variable structuring element based fuzzy morphological operations for single viewpoint omnidirectional images, Pattern Recognition, 40, 3578-3596 (2007) · Zbl 1122.68716
[46] Bloch, I.; Ralescu, A., Directional relative position between objects in image processing: a comparison between fuzzy approaches, Pattern Recognition, 36, 1563-1582 (2003) · Zbl 1055.68134
[47] Pham, T. D., An image restoration by fusion, Pattern Recognition, 34, 2403-2411 (2003) · Zbl 0991.68138
[48] Ramachandran, R. P.; Farrell, K. R.; Ramachandran, R.; Mammone, R. J., Speaker recognition—general classier approaches and data fusion methods, Pattern Recognition, 35, 2801-2821 (2002) · Zbl 1010.68155
[49] Li, X.; Liu, Z.; Leung, K., Detection of vehicles from traffic scenes using fuzzy integrals, Pattern Recognition, 35, 967-980 (2002) · Zbl 0999.68571
[50] Yang, R.; Wang, Z.; Heng, P.; Leung, K., Fuzzified Choquet integral with fuzzy-valued integrand and its application on temperature prediction, IEEE Transactions on Systems, Man, and Cybernetics, 38, 367-380 (2008)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.