×

Three-way decision with co-training for partially labeled data. (English) Zbl 1475.68269

Summary: The theory of three-way decision plays an important role in decision making and knowledge reasoning. However, little attention has been paid to the problem of learning from partially labeled data with three-way decision. In this paper, we propose a three-way co-decision model for partially labeled data. More specifically, the problem of attribute reduction for partially labeled data is first investigated, and two semi-supervised attribute reduction algorithms based on novel confidence discernibility matrix are proposed. Then, a three-way co-decision model is introduced to classify unlabeled data into useful, useless, and uncertain data, and the model is iteratively retrained on the carefully selected useful data to improve its performance. Moreover, we theoretically analyze the effectiveness of the proposed model. The experimental results conducted on UCI data sets demonstrate that the proposed model is promising, and even compares favourably with the single supervised classifier trained on all training data with true labels.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
68T37 Reasoning under uncertainty in the context of artificial intelligence

Software:

WEKA
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] D. Angluin, P. Laird, Learning from noisy examples, Mach. Learn. 2 (1988) 343-370.
[2] A. Blum, T. Mitchell, Combining labeled and unlabeled data with co-training, in: Proceedings of the 11th Annual Conference on Computational Learning Theory, Madison, Wisconsin, USA, 1998, pp. 92-100.
[3] Dai, D.; Li, H. X.; Jia, X. Y.; Zhou, X. Z.; Huang, B.; Liang, S. N., A co-training approach for sequential three-way decisions, Int. J. Mach. Learn. Cybern., 11, 1129-1139 (2020)
[4] Dai, J. H.; Chen, J. L., Feature selection via normative fuzzy information weight with application into tumor classification, Appl. Soft Comput., 92, Article 106299 pp. (2020)
[5] J.H. Dai, H.F. Han, H. Hu, Q.H. Hu, J.H. Zhang, W.T. Wang, DualPOS: A semi-supervised attribute selection approach for symbolic data based on rough set theory, in: B. Cui, N. Zhang, J. Xu, X. Lian, D. Liu (Eds.) Proceedings of the 17th International Conference on Web-Age Information Management, Nanchang, China, 2016, pp. 392-402.
[6] Dai, J. H.; Hu, H.; Wu, W. Z.; Qian, Y. H.; Huang, D. B., Maximal-discernibility-pair-based approach to attribute reduction in fuzzy rough sets, IEEE Trans. Fuzzy Syst., 26, 2174-2187 (2018)
[7] Dai, J. H.; Hu, Q. H.; Zhang, J. H.; Hu, H.; Zheng, N. G., Attribute selection for partially labeled categorical data by rough set approach, IEEE Trans. Cybern., 47, 2460-2471 (2017)
[8] Dai, J. H.; Hu, Q. H.; Hu, H.; Huang, D. B., Neighbor inconsistent pair selection for attribute reduction by rough set approach, IEEE Trans. Fuzzy Syst., 26, 937-950 (2018)
[9] E. Frank, M.A. Hall, I.H. Witten, The WEKA workbench, online appendix for “Data mining: Practical machine learning tools and techniques”, 4th ed., Morgan Kaufmann, 2016.
[10] Hu, Q. H.; Yu, D. R.; Liu, J. F.; Wu, C. X., Neighborhood rough set based heterogeneous feature subset selection, Inf. Sci., 178, 3577-3594 (2008) · Zbl 1154.68466
[11] [1] R. Jensen, S. Vluymans, N. Mac Parthalain, C. Cornelis, Y. Saeys, Semi-supervised fuzzy-rough feature selection, in: Proceedings of the 15th International Conference on Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing, Tianjin, China, 2015, pp. 185-195. · Zbl 1444.68157
[12] Kuo, C. C.; Shieh, H. L., A semi-supervised learning algorithm for data classification, Int. J. Pattern Recognit. Artif. Intell., 29, 1551007 (2015)
[13] Li, B. Y.; Xiao, J. M.; Wang, X. H., Feature selection for partially labeled data based on neighborhood granulation measures, IEEE Access, 7, 37238-37250 (2019)
[14] Lingras, P.; Chen, M.; Miao, D. Q., Semi-supervised rough cost/benefit decisions, Fundam. Inform., 94, 233-244 (2009) · Zbl 1192.68529
[15] Liu, C. H.; Cai, K. C.; Miao, D. Q.; Qian, J., Novel matrix-based approaches to computing minimal and maximal descriptions in covering-based rough sets, Inf. Sci., 539, 312-326 (2020) · Zbl 1474.68360
[16] Liu, K. Y.; Tsang, E. C.C.; Song, J. J.; Yu, H. L.; Chen, X. J.; Yang, X. B., Neighborhood attribute reduction approach to partially labeled data, Granul. Comput., 5, 239-250 (2020)
[17] Liu, K. Y.; Yang, X. B.; Yu, H. L.; Mi, J. S.; Wang, P. X.; Chen, X. J., Rough set based semi-supervised feature selection via ensemble selector, Knowl.-Based Syst., 165, 282-296 (2019)
[18] Miao, D. Q.; Gao, C.; Zhang, N.; Zhang, Z. F., Diverse reduct subspaces based co-training for partially labeled data, Int. J. Approx. Reason., 52, 1103-1117 (2011)
[19] Min, F.; Liu, F. L.; Wen, L. Y.; Zhang, Z. H., Tri-partition cost-sensitive active learning through kNN, Soft Comput., 23, 1557-1572 (2019)
[20] K. Nigam, R. Ghani, Analyzing the effectiveness and applicability of co-training, in: Proceedings of the 9th ACM International Conference on Information and Knowledge Management, McLean, VA, 2000, pp. 86-93.
[21] N.M. Parthalain, R. Jensen, Fuzzy-rough set based semi-supervised learning, in: 2011 IEEE International Conference on Fuzzy Systems, Taipei, Taiwan, 2011, pp. 2465-2472.
[22] Pawlak, Z., Rough sets, Int. J. Comput. Inf. Sci., 11, 341-356 (1982) · Zbl 0501.68053
[23] Z. Pawlak, Rough sets: Theoretical Aspects of Reasoning about Data, Kluwer Academic Publishers, Dordrecht, Netherlands, 1991. · Zbl 0758.68054
[24] Qian, Y. H.; Liang, X. Y.; Lin, G. P.; Guo, Q.; Liang, J. Y., Local multigranulation decision-theoretic rough sets, Int. J. Approx. Reason., 82, 119-137 (2017) · Zbl 1404.68172
[25] Qian, Y. H.; Liang, X. Y.; Wang, Q.; Liang, J. Y.; Liu, B.; Skowron, A.; Yao, Y. Y.; Ma, J. M.; Dang, C. Y., Local rough set: A solution to rough data analysis in big data, Int. J. Approx. Reason., 97, 38-63 (2018) · Zbl 1445.68223
[26] Sengoz, C.; Ramanna, S., Learning relational facts from the web: A tolerance rough set approach, Pattern Recognit. Lett., 67, 130-137 (2015)
[27] A. Skowron, C. Rauszer, The discernibility matrices and functions in information systems, in: R. Slowinski (Ed.) Intelligent decision support: Handbook of applications and advances of the rough sets theory, Springer Netherlands, Dordrecht, 1992, pp. 331-362. · Zbl 0820.68001
[28] Triguero, I.; García, S.; Herrera, F., Self-labeled techniques for semi-supervised learning: Taxonomy, software and empirical study, Knowl. Inf. Syst., 42, 245-284 (2015)
[29] Wang, R.; Chen, D. G.; Kwong, S., Fuzzy-rough-set-based active learning, IEEE Trans. Fuzzy Syst., 22, 1699-1704 (2014)
[30] Yang, B.; Li, J. H., Complex network analysis of three-way decision researches, Int. J. Mach. Learn. Cyb., 11, 973-987 (2020)
[31] Yang, X.; Li, T. R.; Liu, D.; Chen, H. M.; Luo, C., A unified framework of dynamic three-way probabilistic rough sets, Inf. Sci., 420, 126-147 (2017) · Zbl 1436.68362
[32] Yao, Y. Y., Probabilistic rough set approximations, Int. J. Approx. Reason., 49, 255-271 (2008) · Zbl 1191.68702
[33] Yao, Y. Y., Three-way decisions with probabilistic rough sets, Inf. Sci., 180, 341-353 (2010)
[34] Yao, Y. Y., The superiority of three-way decisions in probabilistic rough set models, Inf. Sci., 181, 1080-1096 (2011) · Zbl 1211.68442
[35] Yao, Y. Y., Three-way decisions and cognitive computing, Cogn. Comput., 8, 543-554 (2016)
[36] Yao, Y. Y., Three-way decision and granular computing, Int. J. Approx. Reason., 103, 107-123 (2018) · Zbl 1448.68427
[37] Yao, Y. Y., Three-way conflict analysis: Reformulations and extensions of the Pawlak model, Knowl.-Based Syst., 180, 26-37 (2019)
[38] Yao, Y. Y., Tri-level thinking: models of three-way decision, Int. J. Mach. Learn. Cybern., 11, 947-959 (2020)
[39] Yao, Y. Y., Three-way granular computing, rough sets, and formal concept analysis, Int. J. Approx. Reason., 116, 106-125 (2020) · Zbl 1468.68246
[40] Yao, Y. Y.; Wang, S.; Deng, X. F., Constructing shadowed sets and three-way approximations of fuzzy sets, Inf. Sci., 412-413, 132-153 (2017) · Zbl 1435.68330
[41] Yao, Y. Y.; Wong, S. K.M., A decision theoretic framework for approximating concepts, Int. J. Man-Mach. Stud., 37, 793-809 (1992)
[42] Yao, Y. Y.; Yao, B. X., Covering based rough set approximations, Inf. Sci., 200, 91-107 (2012) · Zbl 1248.68496
[43] Yao, Y. Y.; Zhao, Y., Attribute reduction in decision-theoretic rough set models, Inf. Sci., 178, 3356-3373 (2008) · Zbl 1156.68589
[44] Yao, Y. Y.; Zhao, Y., Discernibility matrix simplification for constructing attribute reducts, Inf. Sci., 179, 867-882 (2009) · Zbl 1162.68704
[45] Yue, X. D.; Chen, Y. F.; Miao, D. Q.; Fujita, H., Fuzzy neighborhood covering for three-way classification, Inf. Sci., 507, 795-808 (2020) · Zbl 1456.62122
[46] Zhang, Q. H.; Xie, Q.; Wang, G. Y., A survey on rough set theory and its applications, CAAI Trans. Intell. Technol., 1, 323-333 (2016)
[47] Zhou, J.; Pedrycz, W.; Gao, C.; Lai, Z. H.; Yue, X. D., Principles for constructing three-way approximations of fuzzy sets: A comparative evaluation based on unsupervised learning, Fuzzy Sets Syst. (2020)
[48] Zhou, Z. H., A brief introduction to weakly supervised learning, Natl. Sci. Rev., 5, 44-53 (2018)
[49] Zhou, Z. H.; Li, M., Semi-supervised learning by disagreement, Knowl. Inf. Syst., 24, 415-439 (2010)
[50] X.J. Zhu, and A.B. Goldberg, Introduction to Semi-supervised Learning, Morgan & Claypool Publishers, Cambridge, MA, USA, 2009. · Zbl 1209.68435
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.