×

zbMATH — the first resource for mathematics

Avoiding optimal mean \(\ell_{2,1}\)-norm maximization-based robust PCA for reconstruction. (English) Zbl 1418.62245
Summary: Robust principal component analysis (PCA) is one of the most important dimension-reduction techniques for handling high-dimensional data with outliers. However, most of the existing robust PCA presupposes that the mean of the data is zero and incorrectly utilizes the average of data as the optimal mean of robust PCA. In fact, this assumption holds only for the squared \(\ell_{2}\)-norm-based traditional PCA. In this letter, we equivalently reformulate the objective of conventional PCA and learn the optimal projection directions by maximizing the sum of projected difference between each pair of instances based on \(\ell_{2,1}\)-norm. The proposed method is robust to outliers and also invariant to rotation. More important, the reformulated objective not only automatically avoids the calculation of optimal mean and makes the assumption of centered data unnecessary, but also theoretically connects to the minimization of reconstruction error. To solve the proposed nonsmooth problem, we exploit an efficient optimization algorithm to soften the contributions from outliers by reweighting each data point iteratively. We theoretically analyze the convergence and computational complexity of the proposed algorithm. Extensive experimental results on several benchmark data sets illustrate the effectiveness and superiority of the proposed method.

MSC:
62H25 Factor analysis and principal components; correspondence analysis
62G35 Nonparametric robustness
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Cai, D., He, X., Hu, Y., Han, J., & Huang, T. (2007). Learning a spatially smooth subspace for face recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE. ,
[2] Candès, E. J., Li, X., Ma, Y., & Wright, J. (2011). Robust principal component analysis?Journal of the ACM, 58, 11. · Zbl 1327.62369
[3] Chang, X., Nie, F., Ma, Z., Yang, Y., & Zhou, X. (2015). A convex formulation for spectral shrunk clustering. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (pp. 2532-2538). Cambridge, MA: MIT Press.
[4] Chang, X., Nie, F., Wang, S., Yang, Y., Zhou, X., & Zhang, C. (2015). Compound rank-##img## projections for bilinear analysis. IEEE Transactions on Neural Networks and Learning Systems, 27, 1502-1513. ,
[5] Chang, X., Nie, F., Yang, Y., & Huang, H. (2014). A convex sparse PCA for feature analysis. arXiv:1411.6233
[6] Dailey, M. N., Joyce, C., Lyons, M. J., Kamachi, M., Ishi, H., Gyoba, J., & Cottrell, G. W. (2010). Evidence and a computational explanation of cultural differences in facial expression recognition. Emotion, 10, 874. ,
[7] De Lathauwer, L. (1997). Signal processing based on multilinear algebra.Leuven: Katholieke Universiteit Leuven.
[8] Ding, C., Zhou, D., He, X., & Zha, H. (2006). R1-PCA: Rotational invariant ##img##-norm principal component analysis for robust subspace factorization. In Proceedings of International Conference on Machine Learning.New York: ACM. ,
[9] Gottumukkal, R., & Asari, V. K. (2004). An improved face recognition technique based on modular PCA approach. Pattern Recognition Letter, 25, 429-436. ,
[10] He, R., Hu, B., Zheng, W., & Kong, X. (2011). Robust principal component analysis based on maximum correntropy criterion. IEEE Transactions on Image Processing, 20, 1485-1494. , · Zbl 1372.94369
[11] Huang, H., & Ding, C. (2008). Robust tensor factorization using ##img## norm. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE.
[12] Jolliffe, I. (2002). Principal component analysis. New York: Wiley. · Zbl 1011.62064
[13] Ke, Q., & Kanade, T. (2005). Robust ##img## norm factorization in the presence of outliers and missing data by alternative convex programming. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE.
[14] Kim, C., & Choi, C.-H. (2007). Image covariance-based subspace method for face recognition. Pattern Recognition, 40, 1592-1604. , · Zbl 1107.68463
[15] Kong, D., Ding, C., & Huang, H. (2011). Robust nonnegative matrix factorization using ##img##-norm. In Proceedings of ACM International Conference on Information and Knowledge Management. New York: ACM. ,
[16] Kwak, N. (2008). Principal component analysis based on ##img##-norm maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 1672-1680. ,
[17] Kwak, N. (2014). Principal component analysis by ##img##-norm maximization. IEEE Transactions on Cybernetics, 44, 594-609. ,
[18] Li, B. N., Yu, Q., Wang, R., Xiang, K., Wang, M., & Li, X. (2016). Block principal component analysis with nongreedy ##img##-norm maximization. IEEE Transactions on Cybernetics, 46, 2543-2547. ,
[19] Li, X., Pang, Y., & Yuan, Y. (2010). L1-norm-based 2dpca. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 40, 1170-1175. ,
[20] Liu, Y., Liu, Y., & Chan, K. C. C. (2010). Multilinear maximum distance embedding via ##img##-norm optimization. In Proceedings of Association for the Advancement of Artificial Intelligence.
[21] Luo, M., Nie, F., Chang, X., Yang, Y., Hauptmann, A. G., & Zheng, Q. (2016). Avoiding optimal mean robust PCA/2DPCA with non-greedy ##img##-norm maximization. In Proceedings of International Joint Conference on Artificial Intelligence. Cambridge, MA: MIT Press.
[22] Markopoulos, P. P., Karystinos, G. N., & Pados, D. A. (2014). Optimal algorithms for-subspace signal processing. IEEE Transactions on Image Processing, 62, 5046-5058. , · Zbl 1394.94375
[23] Martinez, A. M. (1998). The AR face database (CVC Technical Report). West Lafayette, IN: Purdue University.
[24] Meng, D., Zhao, Q., & Xu, Z. (2012). Improve robustness of sparse pca by l 1-norm maximization. Pattern Recognition, 45, 487-497. , · Zbl 1225.68202
[25] Messer, K., Matas, J., Kittler, J., Luettin, J., & Maitre, G. (1999). XM2VTSDB: The extended M2VTS database. In Proceedings of the Second International Conference on Audio and Video-Based Biometric Person Authentication.
[26] Nene, S. A., Nayar, S. K., & Murase, H. (1996). Columbia Object Image Library (COIL-20) (Technical Report CUCS-005-96). New York: Columbia University.
[27] Nie, F., & Huang, H. (2016). Non-greedy ##img##-norm maximization for principal component analysis. arXiv:1603.08293
[28] Nie, F., Huang, H., Cai, X., & Ding, C. H. Q. (2010). Efficient and robust feature selection via joint ##img##-norms minimization. In J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, & A. Culotta (Eds.), Advances in neural information processing systems, 23. Red Hook, NY: Curran.
[29] Nie, F., Huang, H., Ding, C., Luo, D., & Wang, H. (2011). Robust principal component analysis with non-greedy ##img##-norm maximization. In Proceedings of the International Joint Conference on Artificial Intelligence. Cambridge, MA: MIT Press.
[30] Nie, F., Yuan, J., & Huang, H. (2014). Optimal mean robust principal component analysis. In Proceedings of International Conference on Machine Learning.New York: ACM.
[31] Oh, J., & Kwak, N. (2016). Generalized mean for robust principal component analysis. Pattern Recognition, 54, 116-127. ,
[32] Pang, Y., Li, X., & Yuan, Y. (2010). Robust tensor analysis with ##img##-norm. IEEE Transactions on Circuits and Systems for Video Technology, 20, 172-178. ,
[33] Parsons, L., Haque, E., & Liu, H. (2004). Subspace clustering for high dimensional data: a review. SIGKDD Explorations, 6, 90-105. ,
[34] Samaria, F. S., & Harter, A. C. (1994). Parameterisation of a stochastic model for human face identification. In Proceedings of the Second IEEE Workshop on Applications of Computer Vision.Piscataway, NJ: IEEE. ,
[35] Torre, F. D., & Black, M. J. (2001). Robust principal component analysis for computer vision. In Proceedings of International Conference on Computer Vision. Piscataway, NJ: IEEE.
[36] Wang, J. (2016). Generalized 2-D principal component analysis by ##img##-norm for image analysis. IEEE Transactions on Cybernetics, 46, 792-803. ,
[37] Wang, R., Nie, F., Yang, X., Gao, F., & Yao, M. (2015). Robust 2DPCA with non-greedy-norm maximization for image analysis. IEEE Transactions on Cybernetics, 45, 1108-1112. ,
[38] Wang, S., Nie, F., Chang, X., Yao, L., Li, X., & Sheng, Q. Z. (2015). Unsupervised feature analysis with class margin optimization. In Proceedings of Machine Learning and Knowledge Discovery in Databases (pt. I, pp. 383-398). New York: ACM. ,
[39] Wechsler, H., Phillips, J. P., Bruce, V., Soulie, F. F., & Huang, T. S. (2012). Face recognition: From theory to applications. New York: Springer Science & Business Media.
[40] Wright, J., Ganesh, A., Rao, S. R., Peng, Y., & Ma, Y. (2009). Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, & A. Culotta (Eds.), Advances in neural information processing systems, 22. Red Hook, NY: Curran.
[41] Yang, J., Zhang, D., Frangi, A. F., & Yang, J.-Y. (2004). Two-dimensional PCA: A new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 131-137. ,
[42] Zhang, H., Zha, Z.-J., Yang, Y., Yan, S., & Chua, T.-S. (2014). Robust (semi) nonnegative graph embedding. IEEE Transactions on Image Processing, 23, 2996-3012. , · Zbl 1374.94443
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.