Semi-supervised metric learning via topology preserving multiple semi-supervised assumptions. (English) Zbl 1323.68447

Summary: Learning an appropriate distance metric is a critical problem in pattern recognition. This paper addresses the problem of semi-supervised metric learning. We propose a new regularized semi-supervised metric learning (RSSML) method using local topology and triplet constraints. Our regularizer is designed and developed based on local topology, which is represented by local neighbors from the local smoothness, cluster (low density) and manifold information point of view. The regularizer is then combined with the large margin hinge loss on the triplet constraints. In other words, we keep a large margin between different labeled samples, and in the meanwhile, we use the unlabeled samples to regularize it. Then the semi-supervised metric learning method is developed. We have performed experiments on classification using publicly available databases to evaluate the proposed method. To our best knowledge, this is the only method satisfying all the three semi-supervised assumptions, namely smoothness, cluster (low density) and manifold. Experimental results have shown that the proposed method outperforms state-of-the-art semi-supervised distance metric learning algorithms.


68T05 Learning and adaptive systems in artificial intelligence


Full Text: DOI


[1] M. Baghshah, S. Shouraki, Semi-supervised metric learning using pairwise constraints, in: Proceedings of the 21st International Joint Conference on Artificial intelligence, 2009, pp. 1217-1222.
[2] Bar-Hillel, A.; Herts, T.; Shental, N.; Weinshall, D., Learning a Mahalanobis metric from equivalence constraints, Journal of Machine Learning Research, 6, 937-965, (2005) · Zbl 1222.68140
[3] D. Cai, X. He, J. Han, Semi-supervised discriminant analysis, in: IEEE 11th International Conference on Computer Vision, 2007, pp. 1-7.
[4] O. Chapelle, B. Schölkopf, A. Aien, Semi-Supervised Learning, Cambridge, 2006.
[5] Chapelle, O.; Zien, A., Semi-supervised classification by low density separation, Biological Cybernetics, 2005, 57-64, (2004)
[6] Chen, K.; Wang, S. H., Semi-supervised learning via regularized boosting working on multiple semi-supervised assumptions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 1, (2011)
[7] Cox, T. F.; Cox, M. A.A., Multidimensional scaling, (2001), Chapman and Hall · Zbl 1004.91067
[8] Duda, R. O.; Hart, P. E.; Stork, D. G., Pattern classification, (2001), John Wiley and Sons, Inc. · Zbl 0968.68140
[9] A. Frank, A. Asuncion, UCI machine learning repository, 2010. URL \(\langle\)http://archive.ics.uci.edu/ml\(\rangle\).
[10] A. Frome, Y. Singer, F. Sha, J. Malik, Learning globally consistent local distance functions for shape-based image retrieval and classification, in: Proceedings of the IEEE International Conference on Computer Vision, 2007.
[11] Fukunaga, K., Introduction to statistical pattern recognition, (1990), Academic Press · Zbl 0711.62052
[12] Globerson, A.; Roweis, S., Metric learning by collapsing classes, Advances in Neural Information Processing Systems, (2005)
[13] Goldberger, J.; Roweis, S.; Hinton, G.; Salakhutdinov, R., Neighbourhood component analysis, Advances in Neural Information Processing Systems, 17, 513-520, (2005)
[14] He, X.; Niyogi, P., Locality preserving projections, Advances in Neural Information Processing Systems, 16, 153-160, (2004)
[15] Hoi, S. C.; Liu, W.; Chang, S. F., Semi-supervised distance metric learning for collaborative image retrieval and clustering, ACM Transactions on Multimedia Computing, Communications and Applications, 6, 3, (2010)
[16] Hoi, S. C.H.; Lyu, M. R.; Jin, R., A unified log-based relevance feedback scheme for image retrieval, IEEE Transactions on Knowledge and Data Engineering, 18, 4, 524-590, (2006)
[17] Huang, Y.; Xu, D.; Nie, F., Patch distribution compatible semisupervised dimension reduction for face and human gait recognition, IEEE Transactions on Circuits and Systems for Video Technology, 22, 3, 479-488, (2012)
[18] Lee, J. A.; Verleysen, M., Nonlinear dimensionality reduction, (2007), Springer · Zbl 1128.68024
[19] J.E. Lee, R. Jin, A.K. Jain, Rank-based distance metric learning: an application to image retrieval, in: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, 2008.
[20] W. Liu, S. Ma, D. Tao, J. Liu, P. Liu, Semi-supervised sparse metric learning using alternating linearization optimization, in: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2010, pp. 1139-1148.
[21] A. Ma, P. Yuen, Linear dependency modeling for feature fusion, in: 2011 IEEE International Conference on Computer Vision (ICCV), IEEE, 2011, pp. 2041-2048.
[22] Mallapragada, P. K.; Jin, R.; Jain, A. K.; Liu, Y., Semiboostboosting for semi-supervised learning, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 11, 2000-2014, (2009)
[23] Roweis, S. T.; Saul, L. K., Nonlinear dimensionality reduction by local linear embedding, Science, 290, 5500, 2323-2326, (2000)
[24] C. Schuldt, I. Laptev, B. Caputo, Recognizing human actions: a local svm approach, in: Proceedings of the 17th International Conference on Pattern Recognition, (ICPR’04), 2004, pp. 32-36.
[25] Schultz, M.; Joachims, T., Learning a distance metric from relative comparisons, Advances in Neural Information Processing Systems, 16, 41, (2004)
[26] Si, L.; Jin, R.; Steven, C. H.; Hoi, M. R.L., Collaborative image retrieval via regularized metric learning, ACM Transactions on Multimedia Systems, 12, 1, 34-44, (2006)
[27] Soleymani Baghshah, M.; Bagheri Shouraki, S., Non-linear metric learning using pairwise similarity and dissimilarity constraints and the geometrical structure of data, Pattern Recognition, 43, 8, 2982-2992, (2010) · Zbl 1207.68261
[28] Tenenbaum, J. B.; Silva, V.; Langford, J. C., A global geometric framework for nonlinear dimensionality reduction, Science, 290, 5500, 2319-2323, (2000)
[29] Tian, X.; Tao, D.; Hua, X.; Wu, X., Active reranking for web image search, IEEE Transactions on Image Processing, 19, 3, 805-820, (2010) · Zbl 1371.68025
[30] Tian, X.; Tao, D.; Rui, Y., Sparse transfer learning for interactive video search reranking, ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), 8, 3, 26, (2012)
[31] K. Wagstaff, C. Cardie, S. Rogers, S. Schroedl, Constrained k-means clustering with background knowledge, in: Proceedings of the 18th International Conference on Machine Learning, 2001, pp. 577-584.
[32] Q.Y. Wang, P.C. Yuen, G.C. Feng, Semi-supervised metric learning via topology representation, in: European Signal Processing Conference, 2012, pp. 639-643.
[33] Weinberger, K. Q.; Blitzer, J.; Saul, L. K., Distance metric learning for large margin nearest neighbor classification, Advances in Neural Information Processing Systems, 1473-1480, (2006)
[34] Xiang, S.; Nie, F.; Zhang, C., Learning a Mahalanobis distance metric for data clustering and classification, Pattern Recognition, 41, 12, 3600-3612, (2008) · Zbl 1162.68642
[35] Xing, E. P.; Ng, A. Y.; Jordan, M. I.; Russell, S., Distance metric learning with application to clustering with side-information, Advances in Neural Information Processing Systems, 521-528, (2002)
[36] Yang, L.; Jin, R.; Mummert, L.; Sukthankar, R.; Goode, A.; Zheng, B.; Hoi, S. C.; Satyanarayanan, M., A boosting framework for visuality-preserving distance metric learning and its application to medical image retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 1, (2010)
[37] Yeung, D.; Chang, H., A kernel approach for semisupervised metric learning, IEEE Transactions on Neural Networks, 18, 1, 141-149, (2007)
[38] Yu, Y.; Jiang, J.; Zhang, L., Distance metric learning by minimal distance maximization, Pattern Recognition, 44, 3, 639-649, (2011) · Zbl 1209.68497
[39] Jenssen, R., Kernel entropy component analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 5, 847-860, (2010)
[40] Schölkopf, B.; Smola, A.; Müller, K., Nonlinear component analysis as a kernel eigenvalue problem, Neural Computation, 10, 1299-1319, (1998)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.