×

zbMATH — the first resource for mathematics

Bayesian robust principal component analysis with adaptive singular value penalty. (English) Zbl 1446.62166
Summary: Robust principal component analysis (RPCA) has recently seen ubiquitous activity for dimensionality reduction in image processing, visualization and pattern recognition. Conventional RPCA methods model the low-rank component as regularizing each singular value equally. However, in numerous modern applications, each singular value has different physical meaning and should be treated differently. This is one of the main reasons why RPCA techniques cannot work well in dealing with many realistic problems. To solve this problem, a novel hierarchical Bayesian RPCA model with adaptive singular value penalty is proposed. This model enforces the low-rank constraint by introducing an adaptive penalty function on the singular values of the low-rank component. In particular, we impose a hierarchical Exponent-Gamma prior on the singular values of the low-rank component and the Beta-Bernoulli prior on sparsity indicators. The variational Bayesian framework and the Markov chain Monte Carlo-based Bayesian inference are considered for inferring the posteriors of all latent variables involved in low-rank and sparse components. Numerical experiments demonstrate the competitive performance of the proposed model on synthetic and real data.
MSC:
62H25 Factor analysis and principal components; correspondence analysis
68T09 Computational aspects of data analysis and big data
62G35 Nonparametric robustness
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Babacan, SD; Luessi, M.; Molina, R.; Katsaggelos, AK, Sparse Bayesian methods for low-rank matrix estimation, IEEE Trans. Signal Process., 60, 8, 3964-3977 (2012) · Zbl 1393.94670
[2] Babacan, SD; Nakajima, S.; Do, MN, Bayesian group-sparse modeling and variational inference, IEEE Trans. Signal Process., 62, 11, 2906-2921 (2015) · Zbl 1394.94056
[3] Bishop, CM, Pattern Recognition and Machine Learning (2006), Berlin: Springer, Berlin
[4] Cai, J.; Candès, EJ; Shen, Z., A singular value thresholding algorithm for matrix completion, SIAM J. Optim., 20, 4, 1956-1982 (2008) · Zbl 1201.90155
[5] Candès, EJ; Li, X.; Ma, Y.; Wright, J., Robust principal component analysis?, J. ACM, 58, 3, 11 (2009)
[6] Candès, EJ; Wakin, MB; Boyd, SP, Enhancing sparsity by reweighted \(\ell_1\) minimization, J. Fourier Anal. Appl., 14, 5-6, 877-905 (2008) · Zbl 1176.94014
[7] X. Cao, Y. Chen, Q. Zhao, D. Meng, Y. Wang, D. Wang, Z. Xu, Low-rank matrix factorization under general mixture noise distributions, in IEEE International Conference on Computer Vision, (2016), pp. 1493-1501
[8] Cawley, GC; Talbot, NLC, Preventing over-fitting during model selection via Bayesian regularisation of the hyper-parameters, J. Mach. Learn. Res., 8, 841-861 (2007) · Zbl 1222.68160
[9] Chen, F.; Hu, R.; Yu, H.; Wang, S., Reduced set density estimator for object segmentation based on shape probabilistic representation, J. Vis. Commun. Image Represent., 23, 7, 1085-1094 (2012)
[10] Chen, Y.; Cao, X.; Zhao, Q.; Meng, D.; Xu, Z., Denoising hyperspectral image with non-i.i.d. noise structure, IEEE Trans. Cybern., 48, 3, 1054-1066 (2018)
[11] Ding, X.; He, L.; Carin, L., Bayesian robust principal component analysis, IEEE Trans. Image Process., 20, 12, 3419-3430 (2011) · Zbl 1381.62144
[12] Fan, Z.; Yong, X.; Zuo, W.; Jian, Y.; Tang, J.; Lai, Z.; Zhang, D., Modified principal component analysis: an integration of multiple similarity subspace models, IEEE Trans. Neural Netw. Learn. Syst., 25, 8, 1538-1552 (2017)
[13] Gao, J., Robust \(l_1\) principal component analysis and its Bayesian variational inference, Neural Comput., 20, 2, 555-578 (2008) · Zbl 1132.62048
[14] Ghaani Farashahi, A., Cyclic wave packet transform on finite abelian groups of prime order, Int. J. Wavelets Multiresolut. Inf. Process., 12, 6, 1450041 (2014) · Zbl 1364.42039
[15] Ghaani Farashahi, A., Wave packet transforms over finite cyclic groups, Linear Algebra Appl., 489, 75-92 (2016) · Zbl 1327.42038
[16] Giordani, P.; Kiers, HAL, A comparison of three methods for principal component analysis of fuzzy interval data, Comput. Stat. Data Anal., 51, 1, 379-397 (2006) · Zbl 1157.62426
[17] Gu, S.; Xie, Q.; Meng, D.; Zuo, W.; Feng, X.; Zhang, L., Weighted nuclear norm minimization and its applications to low level vision, Int. J. Comput. Vis., 121, 2, 183-208 (2017)
[18] Han, G.; Wang, J.; Cai, X., Background subtraction based on modified online robust principal component analysis, Int. J. Mach. Learn. Cybern., 8, 6, 1839-1852 (2017)
[19] Han, N.; Song, Y.; Song, Z., Bayesian robust principal component analysis with structured sparse component, Comput. Stat. Data Anal., 109, 144-158 (2017) · Zbl 1466.62084
[20] Han, N.; Song, Z., Bayesian multiple measurement vector problem with spatial structured sparsity patterns, Digit. Signal Process., 75, 184-201 (2018)
[21] Han, N.; Song, Z.; Li, Y., Cluster-based image super-resolution via jointly low-rank and sparse representation, J. Vis. Commun. Image Represent., 38, 175-185 (2016)
[22] Hu, Y.; Zhang, D.; Ye, J.; Li, X.; He, X., Fast and accurate matrix completion via truncated nuclear norm regularization, IEEE Trans. Pattern Anal. Mach. Intell., 35, 9, 2117-2130 (2013)
[23] Lee, OY; Lee, JW; Kim, JO, Combining self-learning based super-resolution with denoising for noisy images, J. Vis. Commun. Image Represent., 48, 66-76 (2017)
[24] Z. Lin, M. Chen, Y. Ma, The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055 (2010)
[25] Liu, G.; Lin, Z.; Yan, S.; Sun, J.; Yu, Y.; Ma, Y., Robust recovery of subspace structures by low-rank representation, IEEE Trans. Pattern Anal. Mach. Intell., 35, 1, 171-184 (2013)
[26] Liu, S.; Wu, H.; Huang, Y.; Yang, Y.; Jia, J., Accelerated structure-aware sparse Bayesian learning for 3D electrical impedance tomography, IEEE Trans. Ind. Inform., 15, 9, 5033-5041 (2019)
[27] Liu, S.; Zhang, YD; Shan, T.; Tao, R., Structure-aware Bayesian compressive sensing for frequency-hopping spectrum estimation with missing observations, IEEE Trans. Signal Process., 66, 8, 2153-2166 (2018) · Zbl 1414.94371
[28] Lu, C.; Tang, J.; Yan, S.; Lin, Z., Nonconvex nonsmooth low-rank minimization via iteratively reweighted nuclear norm, IEEE Trans. Image Process., 25, 2, 829-839 (2016) · Zbl 1408.94866
[29] J. Luttinen, A. Ilin, J. Karhunen, Bayesian robust \(pca\) for incomplete data, in International Conference on Independent Component Analysis and Signal Separation, (2009), pp. 66-73 · Zbl 1301.62058
[30] Musa, AB, A comparison of \(\ell_1\)-regularization, PCA, KPCA and ICA for dimensionality reduction in logistic regression, Int. J. Mach. Learn. Cybern., 5, 6, 861-873 (2014)
[31] Nedic, N.; Prsic, D.; Dubonjic, L.; Stojanovic, V.; Djordjevic, V., Optimal cascade hydraulic control for a parallel robot platform by PSO, Int. J. Adv. Manuf. Technol., 72, 5-8, 1085-1098 (2014)
[32] Nedic, N.; Prsic, D.; Fragassa, C.; Stojanovic, V.; Pavlovic, A., Simulation of hydraulic check valve for forestry equipment, Int. J. Heavy Veh. Syst., 24, 3, 260-276 (2017)
[33] Prsic, D.; Nedic, N.; Stojanovic, V., A nature inspired optimal control of pneumatic-driven parallel robot platform, Proc. Inst. Mech. Eng. Part C J. Mech. Eng. Sci., 231, 1, 59-71 (2017)
[34] Serneels, S.; Verdonck, T., Principal component analysis for data containing outliers and missing elements, Comput. Stat. Data Anal., 52, 3, 1712-1727 (2008) · Zbl 1452.62419
[35] Sharma, A.; Paliwal, K.; Imoto, S.; Miyano, S., Principal component analysis using QR decomposition, Int. J. Mach. Learn. Cybern., 4, 6, 679-683 (2013)
[36] Stojanovic, V.; Filipovic, V., Adaptive input design for identification of output error model with constrained output, Circuits Syst. Signal Process., 33, 1, 97-113 (2014)
[37] Stojanovic, V.; Nedic, N.; Prsic, D.; Dubonjic, L., Optimal experiment design for identification of ARX models with constrained output in non-Gaussian noise, Appl. Math. Model., 40, 13-14, 6676-6689 (2016) · Zbl 1465.62136
[38] Toh, KC; Yun, S., An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems, Pac. J. Optim., 6, 3, 615-640 (2011) · Zbl 1205.90218
[39] Wagner, A.; Wright, J.; Ganesh, A.; Zhou, Z.; Mobahi, H.; Ma, Y., Toward a practical face recognition system: robust alignment and illumination by sparse representation, IEEE Trans. Pattern Anal. Mach. Intell., 34, 2, 372-386 (2011)
[40] Yi, S.; Lai, Z.; He, Z.; Liu, Y., Joint sparse principal component analysis, Pattern Recognit., 61, 524-536 (2017) · Zbl 1428.68266
[41] Zhang, Z.; Rao, BD, Sparse signal recovery with temporally correlated source vectors using sparse Bayesian learning, IEEE J. Sel. Top. Signal Process., 5, 5, 912-926 (2011)
[42] Zhao, Q.; Meng, D.; Xu, Z., \(l_1\)-norm low-rank matrix factorization by variational \(B\) ayesian method, IEEE Trans. Neural Netw. Learn. Syst., 26, 4, 825-839 (2015)
[43] Q. Zhao, D. Meng, Z. Xu, W. Zuo, L. Zhang, Robust principal component analysis with complex noise, in International Conference on Machine Learning, (2014), pp. 55-63
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.