Feature selection algorithms in classification problems: an experimental evaluation. (English) Zbl 1116.62069

Salappa, A.; Doumpos, M.; Zopounidis, C.

Optim. Methods Softw. 22, No. 1, 199-212 (2007).

Summary: Feature selection (FS) is a significant topic for the development of efficient pattern recognition systems. FS refers to the selection of the most appropriate subset of features that describes (adequately) a given classification task. The objective of the present paper is to perform a thorough analysis of the performance and efficiency of feature selection algorithms (FSAs). The analysis covers a variety of important issues with respect to the functionality of FSAs, such as: (a) their ability to identify relevant features, (b) the performance of the classification models developed on a reduced set of features, (c) the reduction in the number of features and (d) the interactions between different FSAs with the techniques used to develop a classification model. The analysis considers a variety of FSAs and classification methods.

MSC:

62H30	Classification and discrimination; cluster analysis (statistical aspects)
68T10	Pattern recognition, speech recognition
62-04	Software, source code, etc. for problems pertaining to statistics

Keywords:

feature selection; knowledge discovery; pattern recognition; machine learning

Software:

UCI-ml; C4.5

References:

[1]	Kira, K. and Rendell, L. The feature selection problem: traditional methods and a new algorithm. Proceedings of AAAI-92. Reston: AAAI Press.
[2]	DOI: 10.1016/S0004-3702(97)00063-5 · Zbl 0904.68142 · doi:10.1016/S0004-3702(97)00063-5
[3]	Doak, J. 1992. ”An evaluation of feature selection methods and their application to computer security”. Technical report CSE-92-18, University of California, Department of Computer Science, Davis, CA
[4]	Liu H., Feature Selection for Knowledge Discovery and Data Mining (1998) · Zbl 0908.68127
[5]	Molina, L. C., Belanche, L. and Nebot, A. Feature selection algorithms: A survey and experimental evaluation. Proceedings of the 2002 IEEE International Conference on Data Mining. New York: IEEE Computer Society. · Zbl 1028.68600
[6]	DOI: 10.1109/TC.1977.1674939 · Zbl 0363.68059 · doi:10.1109/TC.1977.1674939
[7]	Devijver P. A., Pattern Recognition: A Statistical Approach (1982) · Zbl 0542.68071
[8]	Ben-Bassat M., Handbook of Statistics (1982) · Zbl 0506.62044
[9]	DOI: 10.1016/0004-3702(94)90084-1 · Zbl 0942.68657 · doi:10.1016/0004-3702(94)90084-1
[10]	Quinlan J. R., C4.5: Programs for Machine Learning (1993)
[11]	Koller, D. and Sahami, M. Toward optimal feature selection. Proceedings of the 13th International Conference on Machine Learning. Edited by: Saitta, L. San Francisco: Morgan Kaufmann.
[12]	Almuallim, H. and Dietterich, T. G. Learning with many irrelevant features. Proceedings of the 9th National Conference on Artificial Intelligenc, Vol. 2, Anaheim: AAAI Press. · Zbl 0942.68657
[13]	Liu, H. and Setiono, R. A probabilistic approach to feature selection: a filter solution. Proceedings of the 13th International Conference on Machine Learning, Edited by: Saitta, L. San Francisco: Morgan Kaufmann.
[14]	Liu, H. and Setiono, R. Scalable feature selection for large sized databases. Proceedings of the 4th World Congress on Expert Systems, San Francisco: Morgan Kaufmann.
[15]	Choubey, S. K., Deogun, J. S., Raghavan, V. V. and Sever, H. A comparison of feature selection algorithms in the context of rough classifiers. Proceedings of the 5th IEEE international Conference on Fuzzy System. New Orleans, LA. Vol. 2,
[16]	DOI: 10.1016/0167-8655(94)90127-9 · doi:10.1016/0167-8655(94)90127-9
[17]	Dash, M. and Liu, H. Hybrid search of feature subsets. Proceedings of the 15th Pacific Rim International Conference on Artificial Intelligence. Edited by: Lee, H. Y. and Motoda, H. Singapore: Springer Verlag.
[18]	DOI: 10.1016/S0004-3702(97)00043-X · Zbl 0904.68143 · doi:10.1016/S0004-3702(97)00043-X
[19]	Stone M., Journal of the Royal Statistical Society B 36 pp 111– (1974)
[20]	DOI: 10.2307/2288636 · Zbl 0543.62079 · doi:10.2307/2288636
[21]	Breiman L., Classification and Regression Trees (1984) · Zbl 0541.62042
[22]	DOI: 10.1007/BF01001956 · Zbl 0501.68053 · doi:10.1007/BF01001956
[23]	Quinlan J. R., Machine Learning 1 pp 81– (1986)
[24]	Moody J., Advances in Neural Information Processing Systems (1992)
[25]	Vapnik V. N., Statistical Learning Theory (1998) · Zbl 0935.62007
[26]	Fukunaga K., Introduction to Statistical Pattern Recognition,, 2. ed. (1990) · Zbl 0711.62052
[27]	Klecka W. R., Discriminant Analysis (1980) · doi:10.4135/9781412983938
[28]	DOI: 10.1002/0471722146 · doi:10.1002/0471722146
[29]	Duda R. O., Pattern Classification,, 2. ed. (2001) · Zbl 0968.68140
[30]	DOI: 10.1016/0893-6080(90)90049-Q · doi:10.1016/0893-6080(90)90049-Q
[31]	Blake C., UCI repository of machine learning data bases (1998)