×

Semi-supervised learning with nuclear norm regularization. (English) Zbl 1316.68119

Summary: Integrating new knowledge sources into various learning tasks to improve their performance has recently become an interesting topic. In this paper we propose a novel semi-supervised learning (SSL) approach, called semi-supervised learning with nuclear norm regularization (SSL-NNR), which can simultaneously handle both sparse labeled data and additional pairwise constraints together with unlabeled data. Specifically, we first construct a unified SSL framework to combine the manifold assumption and the pairwise constraints assumption for classification tasks. Then we provide a modified fixed point continuous algorithm to learn a low-rank kernel matrix that takes advantage of Laplacian spectral regularization. Finally, we develop a two-stage optimization strategy, and present a semi-supervised classification algorithm with enhanced spectral kernel (ESK). Moreover, we also present a theoretical analysis of the proposed ESK algorithm, and derive an easy approach to extend it to out-of-sample data. Experimental results on a variety of synthetic and real-world data sets demonstrate the effectiveness of the proposed ESK algorithm.

MSC:

68T05 Learning and adaptive systems in artificial intelligence

Software:

MNIST; COIL-20; LIBSVM
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Chapelle, O.; Schölkopf, B.; Zien, A., Semi-supervised Learning (2006), The MIT Press: The MIT Press Cambridge, MA
[5] Belkin, M.; Niyogi, P.; Sindhwani, V., Manifold regularization: a geometric framework for learning from labeled and unlabeled examples, Journal of Machine Learning Research, 7, 2399-2434 (2006) · Zbl 1222.68144
[6] Melacci, S.; Belkin, M., Laplacian support vector machines trained in the primal, Journal of Machine Learning Research, 12, 1149-1184 (2011) · Zbl 1280.68182
[8] Yan, R.; Zhang, J.; Yang, J.; Hauptmann, A., A discriminative learning framework with pairwise constraints for video object classification, IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 4, 578-593 (2006)
[11] Erdem, A.; Pelillo, M., Graph transduction as a non-cooperative game, Neural Computation, 24, 3, 700-723 (2012) · Zbl 1238.68121
[26] Zhuang, J.; Tsang, I.; Hoi, S. C.H., A family of simple non-parametric kernel learning algorithms, Journal of Machine Learning Research, 12, 1313-1347 (2011) · Zbl 1280.68223
[31] Hu, E.; Chen, S.; Zhang, D.; Yin, X., Semisupervised kernel matrix learning by kernel propagation, IEEE Transactions on Neural Networks, 21, 11, 1831-1841 (2010)
[36] Wang, F.; Zhang, C., Label propagation through linear neighborhoods, IEEE Transactions on Knowledge and Data Engineering, 20, 1, 55-67 (2008)
[37] Cai, J.; Candès, E. J.; Shen, Z., A singular value thresholding algorithm for matrix completion, SIAM Journal on Optimization, 20, 4, 1956-1982 (2010) · Zbl 1201.90155
[38] Ma, S.; Goldfarb, D.; Chen, L., Fixed point and Bregman iterative methods for matrix rank minimization, Mathematical Programming (Series A), 128, 1, 321-353 (2011) · Zbl 1221.65146
[40] Candès, E. J.; Recht, B., Exact matrix completion via convex optimization, Foundations of Computational Mathematics, 9, 6, 717-772 (2009) · Zbl 1219.90124
[41] Recht, B.; Fazel, M.; Parrilo, P., Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Review, 52, 3, 471-501 (2010) · Zbl 1198.90321
[45] Liu, Z.; Vandenberghe, L., Interior-point method for nuclear norm approximation with application to system identification, SIAM Journal on Matrix Analysis and Applications, 31, 3, 1235-1256 (2010) · Zbl 1201.90151
[49] Barzilai, J.; Borwein, J., Two-point step size gradient methods, IMA Journal of Numerical Analysis, 8, 141-148 (1988) · Zbl 0638.65055
[50] Golub, G. H.; Loan, C. F.V., Matrix computations (1996), Johns Hopkins University Press · Zbl 0865.65009
[57] Georghiades, A. S.; Belhumeur, P. N.; Kriegman, D. J., From few to many: illumination cone models for face recognition under variable lighting and pose, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 6, 643-660 (2001)
[59] Shang, F.; Jiao, L. C.; Liu, Y., Integrating spectral kernel learning and constraints in semi-supervised classification, Neural Processing Letters, 36, 2, 101-115 (2012)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.