×

Online semi-supervised support vector machine. (English) Zbl 1440.68226

Summary: Recently, support vector machine (SVM) has received much attention due to its good performance and wide applicability. As a supervised learning algorithm, the standard SVM uses sufficient labeled data to obtain the optimal decision hyperplane. However, in many practical applications, it is difficult and/or expensive to obtain labeled data. Besides, the standard SVM is a batch learning algorithm. It is inefficient to handle streaming data as the classifier must be retrained from scratch whenever a new data is arrived. In this paper, we consider the online classification of streaming data when only a small portion of data are labeled while a large portion of data are unlabeled. In order to obtain an adaptive solution with relatively low computational complexity, a new form of manifold regularization is proposed. Then, an adaptive and online semi-supervised least square SVM is developed, which well exploits the information of new incoming labeled or unlabeled data to boost learning performance. Simulations on synthetic and real data sets show that the proposed algorithm achieves good classification performance even if there only exist a few labeled data.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
68W27 Online algorithms; streaming algorithms

Software:

COIL-20
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] C. Blake, C. Merz, UCI repository of machine learning databases, Online Available: http://archive.ics.uci.edu/ml/datasets.html; C. Blake, C. Merz, UCI repository of machine learning databases, Online Available: http://archive.ics.uci.edu/ml/datasets.html
[2] Chapelle, O.; Scholkopf, B.; Zien, A., Semi-Supervised Learning, 2 (2006), MIT Press: MIT Press London, UK
[3] Chapelle, O.; Sindhwani, V.; Keerthi, S. S., Optimization techniques for semi-supervised support vector machines, J. Mach. Learn. Res., 9, 2, 203-233 (2008) · Zbl 1225.68158
[4] Chen, M. S.; Ho, T. Y.; Huang, D. Y., Online transductive support vector machines for classification, Proceedings of IEEE International Conference on Information Security and Intelligence Control, 258-261 (2012)
[5] Chen, B.; Zhao, S.; Zhu, P.; Príncipe, J. C., Quantized kernel least mean square algorithm, IEEE Trans. Neural Netw. Learn. Syst., 23, 1, 22-32 (2012)
[6] Chen, B.; Zhao, S.; Zhu, P.; Príncipe, J. C., Quantized kernel recursive least squares algorithm, IEEE Trans. Neural Netw. Learn. Syst., 24, 9, 1484-1491 (2013)
[7] Deng, C. Y., A generalization of the Sherman-Morrison-Woodbury formula, Appl. Math. Lett., 24, 9, 1561-1564 (2011) · Zbl 1241.47003
[8] Forestier, G.; Wemmert, C., Semi-supervised learning using multiple clusterings with limited labeled data, Inf. Sci., 361-362, 20, 48-65 (2016)
[9] Golub, G. H.; Loan, C. F.V., Matrix Computation (2012), JHU Press
[10] Hager, W. W., Updating the inverse of a matrix, SIAM Rev., 31, 2, 221-239 (1989) · Zbl 0671.65018
[11] Hao, Z.; Yu, S.; Yang, X.; Zhao, F.; Hu, R.; Liang, Y., Online ls-svm learning for classification problems based on incremental chunk, Proceedings of International Symposium on Neural Networks, 558-564 (2004)
[12] Hoi, S. C.H.; Jin, R.; Zhu, J.; Lyu, M. R., Semi-supervised SVM batch mode active learning for image retrieval, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1-7 (2008), Anchorage: Anchorage AK
[13] Huang, S.; Li, C., Distributed extreme learning machine for nonlinear learning over network, Entropy, 17, 2, 818-840 (2015)
[14] Huang, G.; Song, S.; Gupta, J. N.; Wu, C., Semi-supervised and unsupervised extreme learning machines, IEEE Trans. Cybern., 44, 12, 2405-2417 (2014)
[15] Li, J.; Cao, Y.; Wang, Y.; Xiao, H., Online learning algorithms for double-weighted least squares twin bounded support vector machines, Neural Process. Lett., 45, 1, 1-21 (2016)
[16] Liu, C. L.; Hsaio, W. H.; Lee, C. H.; Chang, T. H.; Kuo, T. H., Semi-supervised text classification with universum learning, IEEE Trans. Cybern., 46, 2, 462-473 (2017)
[17] Mousavi, A.; Ghidary, S. S.; Karimi, Z., Semi-supervised intrusion detection via online laplacian twin support vector machine, Proceedings of Signal Processing and Intelligent Systems Conference, 138-142 (2015)
[18] Nan, S.; Sun, L.; Chen, B.; Lin, Z.; Toh, K. A., Density-dependent quantized least squares support vector machine for large data sets, IEEE Trans. Neural Netw. Learn. Syst., 28, 1, 94-106 (2017)
[19] Nene, S. A.; Nayar, S. K.; Murase, H., Columbia Object Image Library, (coil-20), Tech. Rep. (1996)
[20] Ohlsson, H.; Ljung, L., Semi-supervised regression and system identification, Three Decades of Progress in Control Science, 343-360 (2010), Springer · Zbl 1234.93117
[21] Schölkopf, B.; Smola, A. J., Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond (2002), MIT Press: MIT Press Cambridge, MA
[22] Shen, P.; Li, C., Distributed information theoretic clustering, IEEE Trans. Signal Process., 62, 13, 3442-3453 (2014) · Zbl 1394.94805
[23] Taghipour, K.; Ng, H. T., Semi-supervised word sense disambiguation using word embeddings in general and specific domains, Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 314-323 (2015)
[24] Tu, E.; Zhang, Y.; Zhu, L.; Yang, J.; Kasabov, N., A graph-based semi-supervised k nearest-neighbor method for nonlinear manifold distributed data classification, Inf. Sci., 367-368, 1, 673-688 (2016) · Zbl 1428.68259
[25] Xu, S.; An, X.; Qiao, X.; Zhu, L.; Li, L., Semi-supervised least-squares support vector regression machines, J. Inf. Comput. Sci., 8, 6, 885-892 (2011)
[26] Yang, X.; Tan, L.; He, L., A robust least squares support vector machine for regression and classification with noise, Neurocomputing, 140, 41-52 (2014)
[27] Yoo, J.; Kim, H. J., Online estimation using semi-supervised least square SVR, Proceedings of IEEE International Conference on System, Man and Cybernetics, 1624-1629 (2014), San Diego: San Diego CA
[28] Zhang, Z.; Zhang, L.; Zhao, M.; Jiang, W.; Li, F.; Li, F., Semi-supervised image classification by nonnegative sparse neighborhood propagation, Proceedings of ACM International Conference on Multimedia Retrieval, 139-146 (2015)
[29] Zhao, M.; Zhan, C.; Wu, Z.; Tang, P., Semi-supervised image classification based on local and global regression, IEEE Signal Process. Lett., 22, 10, 1666-1670 (2015)
[30] Zhou, Z.-H.; Li, M., Semisupervised regression with cotraining-style algorithms, IEEE Trans. Knowl. Data Eng., 19, 11, 1479-1493 (2007)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.