×

Multi-label classification using a fuzzy rough neighborhood consensus. (English) Zbl 1436.68313

Summary: A multi-label dataset consists of observations associated with one or more outcomes. The traditional classification task generalizes to the prediction of several class labels simultaneously. In this paper, we propose a new nearest neighbor based multi-label method. The nearest neighbor approach remains an intuitive and effective way to solve classification problems and popular multi-label classifiers adhering to this paradigm include the MLKNN and IBLR methods. To classify an instance, our proposal derives a consensus among the labelsets of the nearest neighbors based on fuzzy rough set theory. This mathematical framework captures data uncertainty and offers a way to extract a labelset from the dataset that summarizes the information contained in the labelsets of the neighbors. In our experimental study, we compare the performance of our method with five other nearest neighbor based multi-label classifiers using five evaluation metrics commonly used in multi-label classification. Based on the results on both synthetic and real-world datasets, we are able to conclude that our method is a strong competitor to nearest neighbor based multi-label classifiers like MLKNN and IBLR.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
68T37 Reasoning under uncertainty in the context of artificial intelligence

Software:

MultiP-SChlo; ML-KNN
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Beyer, K.; Goldstein, J.; Ramakrishnan, R.; Shaft, U., When is nearest neighbor meaningful?, Proceedings of the International Conference on Database Theory, Springer, 217-235 (1999)
[2] Bhowmick, P.; Basu, A.; Mitra, P.; Prasad, A., Sentence level news emotion analysis in fuzzy multi-label classification framework, Special issue: Natural Lang. Process. Appl., 143 (2010)
[3] Boutell, M.; Luo, J.; Shen, X.; Brown, C., Learning multi-label scene classification, Pattern Recognit., 37, 9, 1757-1771 (2004)
[4] Brinker, K.; Hüllermeier, E., Case-based multilabel ranking, Proceedings of the 20th International joint conference on Artificial Intelligence, 702-707 (2007)
[5] Chen, J.; Tang, Y.; Chen, C.; Fang, B.; Lin, Y.; Shang, Z., Multi-label learning with fuzzy hypergraph regularization for protein subcellular location prediction, IEEE Trans. Nanobiosci., 13, 4, 438-447 (2014)
[6] Cheng, W.; Hüllermeier, E., Combining instance-based learning and logistic regression for multilabel classification, Mach. Learn., 76, 2-3, 211-225 (2009) · Zbl 1470.68091
[7] Cheng, W.; Hüllermeier, E., A simple instance-based approach to multilabel classification using the mallows model, Working Notes of the First International Workshop on Learning from Multi-Label Data, 28-38 (2009)
[8] Chiang, T.; Lo, H.; Lin, S., A ranking-based KNN approach for multi-label classification., ACML, 25, 81-96 (2012)
[9] Cornelis, C.; Jensen, R.; Hurtado, G.; Ślezak, D., Attribute selection with fuzzy decision reducts, Inf. Sci. (Ny), 180, 2, 209-224 (2010) · Zbl 1183.68604
[10] Cornelis, C.; Verbiest, N.; Jensen, R., Ordered weighted average based fuzzy rough sets, Proceedings of the International Conference on Rough Sets and Knowledge Technology, Springer, 78-85 (2010)
[11] Cover, T.; Hart, P., Nearest neighbor pattern classification, IEEE Trans. Inf. Theory, 13, 1, 21-27 (1967) · Zbl 0154.44505
[12] Dubois, D.; Prade, H., Rough fuzzy sets and fuzzy rough sets, Int. J. General Syst., 17, 2-3, 191-209 (1990) · Zbl 0715.04006
[13] Friedman, M., The use of ranks to avoid the assumption of normality implicit in the analysis of variance, J. Am. Stat. Assoc., 32, 200, 675-701 (1937) · JFM 63.1098.02
[14] Fürnkranz, J.; Hüllermeier, E.; Mencía, E.; Brinker, K., Multilabel classification via calibrated label ranking, Mach. Learn., 73, 2, 133-153 (2008) · Zbl 1470.68108
[15] Geng, X.; Tang, Y.; Zhu, Y.; Cheng, G., An improved multi-label classification algorithm BRkNN, J. Inf. Comput. Sci., 11, 16, 5927-5936 (2014)
[16] Gibaja, E.; Ventura, S., Multi-label learning: a review of the state of the art and ongoing research, Wiley Interdiscip. Rev., 4, 6, 411-444 (2014)
[17] Godbole, S.; Sarawagi, S., Discriminative methods for multi-labeled classification, Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer, 22-30 (2004)
[18] Herrera, F.; Charte, F.; Rivera, A.; Del Jesus, M., Multilabel Classification: Problem Analysis, Metrics and Techniques (2016), Springer
[19] Holm, S., A simple sequentially rejective multiple test procedure, Scand. J. Stat., 65-70 (1979) · Zbl 0402.62058
[20] Hu, Q.; Yu, D.; Xie, Z., Neighborhood classifiers, Expert Syst. Appl., 34, 2, 866-876 (2008)
[21] Hu, Q.; Zhang, L.; An, S.; Zhang, D.; Yu, D., On robust fuzzy rough set models, IEEE Trans. Fuzzy Syst., 20, 4, 636-651 (2012)
[22] Huang, K.; Li, Z., A multilabel text classification algorithm for labeling risk factors in SEC form 10-K, ACM Trans. Manage. Inf. Syst. (TMIS), 2, 3, 18 (2011)
[23] Jiang, J.; Tsai, S.; Lee, S., FSKNN: multi-label text categorization based on fuzzy similarity and k nearest neighbors, Expert Syst. Appl., 39, 3, 2813-2821 (2012)
[24] Jing, X.; Wu, F.; Li, Z.; Hu, R.; Zhang, D., Multi-label dictionary learning for image annotation, IEEE Trans. Image Process., 25, 6, 2712-2725 (2016) · Zbl 1408.94290
[25] Kang, F.; Jin, R.; Sukthankar, R., Correlated label propagation with application to multi-label learning, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, 2, 1719-1726 (2006)
[26] Keller, J.; Gray, M.; Givens, J., A fuzzy k-nearest neighbor algorithm, IEEE Trans. Syst. Man Cybern., SMC-15, 4, 580-585 (1985)
[27] Lin, X.; Chen, X., Mr. KNN: soft relevance for multi-label classification, Proceedings of the 19th ACM International Conference on Information and Knowledge Management, ACM, 349-358 (2010)
[28] Liu, H.; Wu, X.; Zhang, S., Neighbor selection for multilabel classification, Neurocomputing, 182, 187-196 (2016)
[29] Liu, S.; Chen, J., A multi-label classification based approach for sentiment classification, Expert Syst. Appl., 42, 3, 1083-1093 (2015)
[30] Mallows, C., Non-null ranking models. i, Biometrika, 44, 1/2, 114-130 (1957) · Zbl 0087.34001
[31] Nanculef, R.; Flaounas, I.; Cristianini, N., Efficient classification of multi-labeled text streams by clashing, Expert Syst. Appl., 41, 11, 5431-5450 (2014)
[32] Pawlak, Z., Rough sets, Int. J. Parallel Program, 11, 5, 341-356 (1982) · Zbl 0501.68053
[33] Radzikowska, A.; Kerre, E., A comparative study of fuzzy rough sets, Fuzzy Sets Syst., 126, 2, 137-155 (2002) · Zbl 1004.03043
[34] Reyes, O.; Morell, C.; Ventura, S., Effective lazy learning algorithm based on a data gravitation model for multi-label learning, Inf. Sci. (Ny), 340, 159-174 (2016)
[35] Spyromitros, E.; Tsoumakas, G.; Vlahavas, I., An empirical study of lazy multilabel classification algorithms, Proceedings of the Hellenic conference on Artificial Intelligence, Springer, 401-406 (2008)
[36] Tomás, J.; Spolaôr, N.; Cherman, E.; Monard, M., A framework to generate synthetic multi-label datasets, Electron Notes Theor. Comput. Sci., 302, 155-176 (2014)
[37] Triguero, I.; Vens, C., Labelling strategies for hierarchical multi-label classification techniques, Pattern Recognit., 56, 170-183 (2016)
[38] Verbiest, N., Fuzzy Rough and Evolutionary Approaches to Instance Selection (2014), Ph.D. thesis, Ghent University
[39] Verbiest, N.; Cornelis, C.; Jensen, R., Fuzzy rough positive region based nearest neighbour classification, Proceedings of the 2012 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), IEEE, 1-7 (2012)
[40] Vluymans, S.; D’eer, L.; Saeys, Y.; Cornelis, C., Applications of fuzzy rough set theory in machine learning: a survey, Fundam. Inform., 142, 1-4, 53-86 (2015) · Zbl 1346.68167
[41] Vluymans, S.; Sánchez Tarragó, D.; Saeys, Y.; Cornelis, C.; Herrera, F., Fuzzy multi-instance classifiers, IEEE Trans. Fuzzy Syst., 24, 6, 1395-1409 (2016)
[42] Vluymans, S.; Sánchez Tarragó, D.; Saeys, Y.; Cornelis, C.; Herrera, F., Fuzzy rough classifiers for class imbalanced multi-instance data, Pattern Recognit, 53, 36-45 (2016)
[43] Wang, H.; Ding, C.; Huang, H., Multi-label classification: Inconsistency and class balanced k-nearest neighbor, Proceedings of the 24th AAAI Conference on Artificial Intelligence, 1264-1266 (2010)
[44] Wang, X.; Zhang, W.; Zhang, Q.; Li, G., MultiP-SChlo: multi-label protein subchloroplast localization prediction with chouâ;;s pseudo amino acid composition and a novel multi-label classifier, Bioinformatics, 31, 16, 2639-2645 (2015)
[45] Wilcoxon, F., Individual comparisons by ranking methods, Biometrics Bull., 1, 6, 80-83 (1945)
[46] Wu, X.; Kumar, V.; Quinlan, J.; Ghosh, J.; Yang, Q.; Motoda, H.; McLachlan, G.; Ng, A.; Liu, B.; Philip, S.; Zhou, Z.; Steinbach, M.; Hand, D.; Steinberg, D., Top 10 algorithms in data mining, Knowl. Inf. Syst., 14, 1, 1-37 (2008)
[47] Xia, X.; Yang, X.; Li, S.; Wu, C.; Zhou, L., RW.KNN: a proposed random walk knn algorithm for multi-label classification, Proceedings of the 4th Workshop on Workshop for Ph.D. Students in Information & Knowledge Management, ACM, 87-90 (2011)
[48] Xu, J., An empirical comparison of weighting functions for multi-label distance-weighted k-nearest neighbour method, Proceedings of the First International Conference on Artificial Intelligence, Soft Computing and Applications (AIAA2011), 13-20 (2011)
[49] Xu, J.; Jagadeesh, V.; Manjunath, B., Multi-label learning with fused multimodal bi-relational graph, IEEE Trans. Multimedia, 16, 2, 403-412 (2014)
[50] Yager, R., On ordered weighted averaging aggregation operators in multicriteria decisionmaking, IEEE Trans. Syst. Man Cybern., 18, 1, 183-190 (1988) · Zbl 0637.90057
[51] Yao, Y., Relational interpretations of neighborhood operators and rough set approximation operators, Inf. Sci. (Ny), 111, 1-4, 239-259 (1998) · Zbl 0949.68144
[52] Younes, Z.; Abdallah, F.; Denœux, T., Multi-label classification algorithm derived from k-nearest neighbor rule with label dependencies, Proceedings of the 2008 16th European Signal Processing Conference, IEEE, 1-5 (2008)
[53] Younes, Z.; Abdallah, F.; Denœux, T., An evidence-theoretic k-nearest neighbor rule for multi-label classification, Proceedings of the International Conference on Scalable Uncertainty Management, Springer, 297-308 (2009)
[54] Younes, Z.; Abdallah, F.; Denœux, T., Evidential multi-label classification approach to learning from data with imprecise labels, Proceedings of the International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, Springer, 119-128 (2010)
[55] Younes, Z.; Abdallah, F.; Denœux, T., Fuzzy multi-label learning under veristic variables, Proceedings of the 2010 IEEE International Conference on Fuzzy Systems (FUZZ), 1-8 (2010), IEEE
[56] Younes, Z.; Abdallah, F.; Denœux, T.; Snoussi, H., A dependent multilabel classification method derived from the k-nearest neighbor rule, EURASIP J. Adv. Signal Process. (2011)
[57] Yu, Y.; Pedrycz, W.; Miao, D., Multi-label classification by exploiting label correlations, Expert Syst. Appl., 41, 6, 2989-3004 (2014)
[58] Zadeh, L., Fuzzy sets, Inf. Control, 8, 3, 338-353 (1965) · Zbl 0139.24606
[59] Zhang, M.; Zhou, Z., ML-KNN: a lazy learning approach to multi-label learning, Pattern Recognit., 40, 7, 2038-2048 (2007) · Zbl 1111.68629
[60] Zhang, M.; Zhou, Z., A review on multi-label learning algorithms, IEEE Trans. Knowl. Data Eng., 26, 8, 1819-1837 (2014)
[61] Zhang, S., Shell-neighbor method and its application in missing data imputation, Appl. Intell., 35, 1, 123-133 (2011)
[62] Zhou, Z.; Zhang, M.; Huang, S.; Li, Y., Multi-instance multi-label learning, Artif. Intell., 176, 1, 2291-2320 (2012) · Zbl 1238.68139
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.