A unified framework for semi-supervised dimensionality reduction. (English) Zbl 1154.68501

Song, Yangqiu; Nie, Feiping; Zhang, Changshui; Xiang, Shiming

Pattern Recognition 41, No. 9, 2789-2799 (2008).

Summary: In practice, many applications require a dimensionality reduction method to deal with the partially labeled problem. In this paper, we propose a semi-supervised dimensionality reduction framework, which can efficiently handle the unlabeled data. Under the framework, several classical methods, such as principal component analysis, linear discriminant analysis, maximum margin criterion, locality preserving projections and their corresponding kernel versions can be seen as special cases. For high-dimensional data, we can give a low-dimensional embedding result for both discriminating multi-class sub-manifolds and preserving local manifold structure. Experiments show that our algorithms can significantly improve the accuracy rates of the corresponding supervised and unsupervised approaches.

MSC:

68T10	Pattern recognition, speech recognition
68T05	Learning and adaptive systems in artificial intelligence

Keywords:

dimensionality reduction; discriminant analysis; manifold analysis; semi-supervised learning

Software:

UCI-ml; KPCA plus LDA

References:

[1]	He, X.; Yan, S.; Hu, Y.; Niyogi, P.; Zhang, H., Face recognition using laplacianfaces, IEEE Trans. Pattern Anal. Mach. Intell., 27, 3, 328-340 (2005)
[2]	Belhumeur, P. N.; Hespanha, J. P.; Kriegman, D. J., Fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell., 19, 7, 711-720 (1997)
[3]	Li, H.; Jiang, T.; Zhang, K., Efficient and robust feature extraction by maximum margin criterion, IEEE Trans. Neural Networks, 17, 1, 157-165 (2006)
[5]	Yang, J.; Frangi, A. F.; Yang, J. Y.; Zhang, D.; Jin, Z., KPCA plus LDA: A complete kernel fisher discriminant framework for feature extraction and recognition, IEEE Trans. Pattern Anal. Mach. Intell., 27, 2, 230-244 (2005)
[8]	Fukunaga, K., Introduction to Statistical Pattern Recognition (1990), Academic Press: Academic Press Boston, MA · Zbl 0711.62052
[11]	Tiknonov, A. N.; Arsenin, V. Y., Solutions of Ill-posed Problems (1977), Wiley: Wiley Washington, DC
[12]	Billings, S. A.; Lee, K. L., Nonlinear fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm, Neural Networks, 15, 2, 263-270 (2002)
[13]	Hastie, T. J.; Buja, A.; Tibshirani, R., Penalized discriminant analysis, Ann. Statist., 23, 1, 73-102 (1995) · Zbl 0821.62031
[14]	Belkin, M.; Niyogi, P.; Sindhwani, V., Manifold regularization: a geometric framework for learning from labeled and unlabeled examples, J. Mach. Learning Res., 1, 1, 1-48 (2006) · Zbl 1222.68144
[15]	Baudat, G.; Anouar, F., Generalized discriminant analysis using a kernel approach, Neural Comput., 12, 10, 2385-2404 (2000)
[25]	Georghiades, A.; Belhumeur, P.; Kriegman, D., From few to many: Illumination cone models for face recognition under variable lighting and pose, IEEE Trans. Pattern Anal. Mach. Intell., 23, 6, 643-660 (2001)
[26]	Graham, D.; Allinson, N., Characterizing virtual eigensignatures for general purpose face recognition, Face Recognition: From Theory to Appl., 163, 446-456 (1998)