×

Typical reconstruction performance for distributed compressed sensing based on \(\ell_{2,1} \)-norm regularized least square and Bayesian optimal reconstruction: influences of noise. (English) Zbl 1456.94019

Summary: A signal model called joint sparse model 2 (JSM-2) or the multiple measurement vector problem, in which all sparse signals share their support, is important for dealing with practical signal processing problems. In this paper, we investigate the typical reconstruction performance of noisy measurement JSM-2 problems for \({{\ell}_{2,1}}\)-norm regularized least square reconstruction and the Bayesian optimal reconstruction scheme in terms of mean square error. Employing the replica method, we show that these schemes, which exploit the knowledge of the sharing of the signal support, can recover the signals more precisely as the number of channels increases. In addition, we compare the reconstruction performance of two different ensembles of observation matrices: one is composed of independent and identically distributed random Gaussian entries and the other is designed so that row vectors are orthogonal to one another. As reported for the single-channel case in earlier studies, our analysis indicates that the latter ensemble offers better performance than the former ones for the noisy JSM-2 problem. The results of numerical experiments with a computationally feasible approximation algorithm we developed for this study agree with the theoretical estimation.

MSC:

94A12 Signal theory (characterization, reconstruction, filtering, etc.)
62F15 Bayesian inference
62H12 Estimation in multivariate analysis
68W40 Analysis of algorithms

Software:

Yall1
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Candes E J and Wakin M B 2008 An introduction to compressive sampling IEEE Signal Process. Mag.25 21-30 · doi:10.1109/MSP.2007.914731
[2] Donoho D L 2006 Compressed sensing IEEE Trans. Inf. Theory52 1289-1306 · Zbl 1288.94016 · doi:10.1109/TIT.2006.871582
[3] Candes E J and Tao T 2005 Decoding by linear programming IEEE Trans. Inf. Theory51 4203-15 · Zbl 1264.94121 · doi:10.1109/TIT.2005.858979
[4] Candes E J, Romberg J and Tao T 2006 Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information IEEE Trans. Inf. Theory52 489-509 · Zbl 1231.94017 · doi:10.1109/TIT.2005.862083
[5] Baron D, Wakin M B, Duarte M F, Sarvotham S and Baraniuk R G 2006 Distributed compressed sensing Technical Report TREE-0612 (arXiv: 0901.3403)
[6] Cotter S F, Rao B D, Engan K and Kreutz-Delgado K 2005 Sparse solutions to linear inverse problems with multiple measurement vectors IEEE Trans. Signal Process.53 2477-88 · Zbl 1372.65123 · doi:10.1109/TSP.2005.849172
[7] Chen J and Huo X 2006 Theoretical results on sparse representations of multiple-measurement vectors IEEE Trans. Signal Process.54 4634-43 · Zbl 1375.94051 · doi:10.1109/TSP.2006.881263
[8] Baraniuk R G, Cevher V, Duarte M F and Hegde C 2010 Model-based compressive sensing IEEE Trans. Inf. Theory56 1982-2001 · Zbl 1366.94215 · doi:10.1109/TIT.2010.2040894
[9] Lee K, Bresler Y and Junge M 2012 Subspace methods for joint sparse recovery IEEE Trans. Inf. Theory58 3613-41 · Zbl 1365.94179 · doi:10.1109/TIT.2012.2189196
[10] Kim J M, Lee O K and Ye J C 2012 Compressive music: revisiting the link between compressive sensing and array signal processing IEEE Trans. Inf. Theory58 278-301 · Zbl 1365.94079 · doi:10.1109/TIT.2011.2171529
[11] Le Roux J, Boufounos P T, Kang K and Hershey J R 2013 Source localization in reverberant environments using sparse optimization IEEE Int. Conf. on Acoustics, Speech and Signal Processing pp 4310-4 · doi:10.1109/ICASSP.2013.6638473
[12] He Z, Cichocki A, Zdunek R and Cao J 2008 CG-M-FOCUSS and its application to distributed compressed sensing Advances in Neural Networks—ISNN 2008(Lecture Notes in Computer Science vol 5263) (Berlin: Springer) pp 237-45 · doi:10.1007/978-3-540-87732-5_27
[13] Zhang W, Ma C, Wang W, Liu Y and Zhang L 2010 Side information based orthogonal matching pursuit in distributed compressed sensing IEEE Int. Conf. on Network Infrastructure and Digital Content pp 80-4 · doi:10.1109/ICNIDC.2010.5657901
[14] Krzakala F, Mézard M, Sausset F, Sun Y F and Zdeborová L 2012 Statistical-physics-based reconstruction in compressed sensing Phys. Rev. X 2 021005 · doi:10.1103/PhysRevX.2.021005
[15] Kabashima Y and Vehkaperä M 2014 Signal recovery using expectation consistent approximation for linear observations IEEE Int. Symp. on Information Theory pp 226-30 · doi:10.1109/ISIT.2014.6874828
[16] Ziniel J and Schniter P 2013 Efficient high-dimensional inference in the multiple measurement vector problem IEEE Trans. Signal Process.61 340-54 · Zbl 1393.94527 · doi:10.1109/TSP.2012.2222382
[17] Gribonval R, Rauhut H, Schnass K and Vandergheynst P 2008 Atoms of all channels, unite! average case analysis of multi-channel sparse recovery using greedy algorithms J. Fourier Anal. Appl.14 655-87 · Zbl 1181.94045 · doi:10.1007/s00041-008-9044-y
[18] Eldar Y C and Rauhut H 2010 Average case analysis of multichannel sparse recovery using convex relaxation IEEE Trans. Inf. Theory56 505-19 · Zbl 1366.94095 · doi:10.1109/TIT.2009.2034789
[19] Donoho D L, Johnstone I and Montanari A 2013 Accurate prediction of phase transitions in compressed sensing via a connection to minimax denoising IEEE Trans. Inf. Theory59 3396-433 · Zbl 1364.94092 · doi:10.1109/TIT.2013.2239356
[20] Vehkaperä M, Chatterjee S and Skoglund M 2011 Analysis of MMSE estimation for compressive sensing of block sparse signals IEEE Information Theory Workshop pp 553-7 · doi:10.1109/ITW.2011.6089563
[21] Dotsenko V 2001 Introduction to the Replica Theory of Disordered Statistical Systems (Cambridge: Cambridge University Press) · Zbl 0999.82001
[22] Nishimori H 2001 Statistical Physics of Spin Glasses and Information Processing vol 187 (Oxford: Oxford University Press) · Zbl 1103.82002 · doi:10.1093/acprof:oso/9780198509417.001.0001
[23] Murayama T and Davis P 200 Statistical mechanics of sensing and communications: insights and techniques J. Phys.: Conf. Ser.95 012010 · doi:10.1088/1742-6596/95/1/012010
[24] Takeda K, Uda S and Kabashima Y 2006 Analysis of CDMA systems that are characterized by eigenvalue spectrum Europhys. Lett.76 1193 · doi:10.1209/epl/i2006-10380-5
[25] Sakata A and Kabashima Y 2013 Statistical mechanics of dictionary learning Europhys. Lett.103 28008 · doi:10.1209/0295-5075/103/28008
[26] Kabashima Y, Wadayama T and Tanaka T 2009 A typical reconstruction limit for compressed sensing based on lp-norm minimization J. Stat. Mech. L09003 · doi:10.1088/1742-5468/2009/09/L09003
[27] Mézard M and Montanari A 2009 Information, Physics, and Computation (Oxford: Oxford University Press) · Zbl 1163.94001 · doi:10.1093/acprof:oso/9780198570837.001.0001
[28] Shiraki Y and Kabashima Y 2015 Typical reconstruction limits for noiseless distributed compressed sensing based on ℓ2,1-norm minimization and bayesian optimal reconstruction J. Stat. Mech. P05029 · Zbl 1456.94018 · doi:10.1088/1742-5468/2015/05/P05029
[29] Deng W, Yin W and Zhang Y 2013 Group sparse optimization by alternating direction method Proc. SPIE8858 88580R · doi:10.1117/12.2024410
[30] Vehkaperä M, Kabashima Y and Chatterjee S 2014 Analysis of regularized LS reconstruction and random matrix ensembles in compressed sensing IEEE Int. Symp. on Information Theory pp 3185-9 · doi:10.1109/ISIT.2014.6875422
[31] Hubbard J 1959 Calculation of partition functions Phys. Rev. Lett.3 77-8 · doi:10.1103/PhysRevLett.3.77
[32] Donoho D L, Maleki A and Montanari A 2009 Message-passing algorithms for compressed sensing Proc. Natl Acad. Sci. USA106 18914-9 · doi:10.1073/pnas.0909892106
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.