×

Neighborhood rough set based heterogeneous feature subset selection. (English) Zbl 1154.68466

Summary: Feature subset selection is viewed as an important preprocessing step for pattern recognition, machine learning and data mining. Most of researches are focused on dealing with homogeneous feature selection, namely, numerical or categorical features. In this paper, we introduce a neighborhood rough set model to deal with the problem of heterogeneous feature subset selection. As the classical rough set model can just be used to evaluate categorical features, we generalize this model with neighborhood relations and introduce a neighborhood rough set model. The proposed model will degrade to the classical one if we specify the size of neighborhood zero. The neighborhood model is used to reduce numerical and categorical features by assigning different thresholds for different kinds of attributes. In this model the sizes of the neighborhood lower and upper approximations of decisions reflect the discriminating capability of feature subsets. The size of lower approximation is computed as the dependency between decision and condition attributes. We use the neighborhood dependency to evaluate the significance of a subset of heterogeneous features and construct forward feature subset selection algorithms. The proposed algorithms are compared with some classical techniques. Experimental results show that the neighborhood model based method is more flexible to deal with heterogeneous data.

MSC:

68T05 Learning and adaptive systems in artificial intelligence

Software:

UCI-ml
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Bhatt, R. B.; Gopal, M., On fuzzy-rough sets approach to feature selection, Pattern Recognition Letters, 26, 965-975 (2005)
[2] Chen, D. G.; Wang, C. Z.; Hu, Q. H., A new approach to attribute reduction of consistent and inconsistent covering decision systems with covering rough sets, Information Sciences, 177, 3500-3518 (2007) · Zbl 1122.68131
[3] Ching, J. Y.; Wong, A. K.C.; Chan, K. C.C., class-dependent discretization for inductive learning from continuous and mixed-mode data, IEEE Transactions on PAMI, 17, 641-651 (1995)
[4] Dash, M.; Liu, H., Consistency-based search in feature selection, Artificial Intelligence, 151, 155-176 (2003) · Zbl 1082.68791
[5] U. Fayyad, K. Irani, Discretizing continuous attributes while learning Bayesian networks, in: Proc. 13th International Conference on Machine Learning, Morgan Kaufmann, 1996, pp. 157-165.; U. Fayyad, K. Irani, Discretizing continuous attributes while learning Bayesian networks, in: Proc. 13th International Conference on Machine Learning, Morgan Kaufmann, 1996, pp. 157-165.
[6] M. Hall, Correlation based feature selection for machine learning, Ph.D. Thesis, University of Waikato, Department of Computer Science, 1999.; M. Hall, Correlation based feature selection for machine learning, Ph.D. Thesis, University of Waikato, Department of Computer Science, 1999.
[7] M. Hall, Correlation-based Feature selection for discrete and numeric class machine learning, in: Proc. 17th ICML. CA, 2000, pp. 359-366.; M. Hall, Correlation-based Feature selection for discrete and numeric class machine learning, in: Proc. 17th ICML. CA, 2000, pp. 359-366.
[8] Q.H. Hu, X.D. Li, D.R. Yu, Analysis on classification performance of rough set based reducts, in: Q. Yang, G. Webb (Eds.), PRICAI 2006, LNAI, vol. 4099, Springer-Verlag, Berlin, Heidelberg, 2006, pp. 423-433.; Q.H. Hu, X.D. Li, D.R. Yu, Analysis on classification performance of rough set based reducts, in: Q. Yang, G. Webb (Eds.), PRICAI 2006, LNAI, vol. 4099, Springer-Verlag, Berlin, Heidelberg, 2006, pp. 423-433.
[9] Hu, Q. H.; Yu, D. R.; Xie, Z. X.; Liu, J. F., Fuzzy probabilistic approximation spaces and their information measures, IEEE Transactions on Fuzzy Systems, 14, 191-201 (2006)
[10] Hu, Q. H.; Yu, D. R.; Xie, Z. X., Information-preserving hybrid data reduction based on fuzzy-rough techniques, Pattern Recognition Letters, 27, 414-423 (2006)
[11] Hu, Q. H.; Yu, D. R.; Xie, Z. X., Neighborhood classifiers, Expert Systems with Applications, 34, 866-876 (2008)
[12] Hu, Q. H.; Liu, J. F.; Yu, D. R., Mixed feature selection based on granulation and approximation, Knowledge-Based Systems, 21, 294-304 (2008)
[13] Jensen, R.; Shen, Q., Semantics-preserving dimensionality reduction: rough and fuzzy-rough-based approaches, IEEE Transactions of Knowledge and Data Engineering, 16, 1457-1471 (2004)
[14] Jensen, R.; Shen, Q., Fuzzy-rough sets assisted attribute selection, IEEE Transactions on Fuzzy Systems, 15, 1, 73-89 (2007)
[15] Jin, W.; Tung, Anthony K. H.; Han, J.; Wang, W., Ranking outliers using symmetric neighborhood relationship, PAKDD, 577-593 (2006)
[16] Li, Y.; Shiu, S. C.K.; Pal, S. K., Combining feature reduction and case selection in building CBR classifiers, IEEE Transactions on Knowledge and Data Engineering, 18, 3, 415-429 (2006)
[17] T.Y. Lin, Neighborhood systems and approximation in database and knowledge base systems, in: Proceedings of the Fourth International Symposium on Methodologies of Intelligent Systems, Poster Session, October 12-15, 1989, pp. 75-86.; T.Y. Lin, Neighborhood systems and approximation in database and knowledge base systems, in: Proceedings of the Fourth International Symposium on Methodologies of Intelligent Systems, Poster Session, October 12-15, 1989, pp. 75-86.
[18] Lin, T. Y., Granulation and Nearest Neighborhoods: Rough Set Approach. Granulation and Nearest Neighborhoods: Rough Set Approach, Granular Computing: An Emerging Paradigm (2001), Physica-Verlag: Physica-Verlag Heidelberg, Germany, pp. 125-142 · Zbl 0986.68144
[19] T.Y. Lin, Neighborhood systems: mathematical models of information granulations, in: 2003 IEEE International Conference on Systems, Man & Cybernetics, Washington, DC, USA, October 5-8, 2003.; T.Y. Lin, Neighborhood systems: mathematical models of information granulations, in: 2003 IEEE International Conference on Systems, Man & Cybernetics, Washington, DC, USA, October 5-8, 2003.
[20] Liu, H.; Yu, L., Towards integrating feature selection algorithms for classification and clustering, IEEE Transactions on Knowledge and Data Engineering, 17, 491-502 (2005)
[21] Muni, D. P.; Pal, N. R.; Das, J., Genetic programming for simultaneous feature selection and classifier design, IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, 36, 1, 106-117 (2006)
[22] Neumann, J.; Schnorr, C.; Steidl, G., Combined SVM-based feature selection and classification, Machine Learning, 61, 129-150 (2005) · Zbl 1137.90643
[23] D.J. Newman, S. Hettich, C.L. Blake, C.J. Merz, UCI Repository of Machine Learning Databases, University of California, Department of Information and Computer Science, Irvine, CA, 1998. <http://www.ics.uci.edu/ mlearn/MLRepository.html>.; D.J. Newman, S. Hettich, C.L. Blake, C.J. Merz, UCI Repository of Machine Learning Databases, University of California, Department of Information and Computer Science, Irvine, CA, 1998. <http://www.ics.uci.edu/ mlearn/MLRepository.html>.
[24] Pawlak, Z., Rough Sets, Theoretical Aspects of Reasoning About Data (1991), Kluwer Academic Publishers: Kluwer Academic Publishers Dordrecht · Zbl 0758.68054
[25] Pawlak, Z.; Skowron, A., Rough Sets: Some Extensions, Information Sciences, 177, 28-40 (2007) · Zbl 1142.68550
[26] Randall Wilson, D.; Martinez, Tony R., Improved heterogeneous distance functions, Journal of Artificial Intelligence Research, 6, 1-34 (1997) · Zbl 0894.68118
[27] Shin, H.; Cho, S., Invariance of neighborhood relation under input space to feature space mapping, Pattern Recognition Letters, 26, 707-718 (2005)
[28] D. Slezak, Approximate reducts in decision tables, in: Proceedings of IPMU’ 96, Granada, Spain, 1996, pp. 1159-1164.; D. Slezak, Approximate reducts in decision tables, in: Proceedings of IPMU’ 96, Granada, Spain, 1996, pp. 1159-1164.
[29] Slowinski, R.; Vanderpooten, D., A generalized definition of rough approximations based on similarity, IEEE Transactions on Knowledge and Data Engineering, 12, 331-336 (2000)
[30] Skowron, A.; Stepaniuk, J., Tolerance approximation spaces, Fundamenta Informaticae, 27, 245-253 (1996) · Zbl 0868.68103
[31] Shen, Q.; Jensen, R., Selecting informative features with fuzzy-rough sets and its application for complex systems monitoring, Pattern recognition, 37, 1351-1363 (2004) · Zbl 1070.68600
[32] Stefanowski, J., On rough set based approaches to induction of decision rules, (Polkowski, L.; Skowron, A., Rough Sets in Knowledge Discovery (1998), Physica-Verlag: Physica-Verlag Heidelberg, Germany), 501-529 · Zbl 0927.68094
[33] Swiniarski, R. W.; Skowron, A., Rough set methods in feature selection and recognition, Pattern Recognition Letters, 24, 833-849 (2003) · Zbl 1053.68093
[34] Tang, W. Y.; Mao, K. Z., Feature selection algorithm for mixed data with both nominal and continuous features, Pattern Recognition Letters, 28, 5, 563-571 (2007)
[35] Wang, H., Nearest neighbors by neighborhood counting, IEEE Transactions on PAMI, 28, 942-953 (2006)
[36] Yao, Y. Y., Relational interpretations of neighborhood operators and rough set approximation operators, Information Sciences, 111, 239-259 (1998) · Zbl 0949.68144
[37] Yeung, D. S.; Chen, D. G.; Tsang, E. C.C.; Lee, J. W.T.; Wang, X. Z., On the generalization of fuzzy rough sets, IEEE Transactions on Fuzzy Systems, 13, 3, 343-361 (2005)
[38] Yu, D. R.; Hu, Q. H.; Bao, W., Combining rough set methodology and fuzzy clustering for knowledge discovery from quantitative data, Proceedings of the CSEE, 24, 6, 205-210 (2004)
[39] L. Yu, H. Liu, Efficiently handling feature redundancy in high-dimensional data, in: Proceedings of The Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-03), Washington, DC, August, 2003, pp. 685-690.; L. Yu, H. Liu, Efficiently handling feature redundancy in high-dimensional data, in: Proceedings of The Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-03), Washington, DC, August, 2003, pp. 685-690.
[40] Yu, L.; Liu, H., Efficient feature selection via analysis of relevance and redundancy, Journal of Machine Learning Research, 5, 1205-1224 (2004) · Zbl 1222.68340
[41] Zhu, Z. X.; Ong, Y. S.; Dash, M., Wrapper-filter feature selection algorithm using a memetic framework, IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, 37, 70-76 (2007)
[42] Zadeh, L., Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic, Fuzzy Sets and Systems, 19, 111-127 (1997) · Zbl 0988.03040
[43] Zadeh, L., Fuzzy logic equals computing with words, IEEE Transactions on Fuzzy Systems, 4, 2, 103-111 (1996)
[44] Zhong, N.; Dong, J.; Ohsuga, S., Using rough sets with heuristics for feature selection, Journal of Intelligent Information Systems, 16, 3, 199-214 (2001) · Zbl 0994.68149
[45] Zhu, W., Generalized rough sets based on relations, Information Sciences, 177, 4997-5011 (2007) · Zbl 1129.68088
[46] Zhu, W.; Wang, F.-Y., Reduction and axiomization of covering generalized rough sets, Information Sciences, 152, 217-230 (2003) · Zbl 1069.68613
[47] Ziarko, W., Variable precision rough sets model, Journal of Computer and System Sciences, 46, 1, 39-59 (1993) · Zbl 0764.68162
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.