×

Optimal prediction in the linearly transformed spiked model. (English) Zbl 1441.62158

The authors consider the linearly transformed spiked model \(Y_i=A_iX_i+\epsilon_i\), \(i=1,\dots,n\). Here, the observations \(Y_i\) are noisy linear transforms of unobserved signals of interest \(X_i\).
The observed transform matrices \(A_i\) reduce the dimension of the signal \(X_i\in \mathbb{R}^p\) to a possibly observation-dependent dimension \(q_i\leq p,\) thus \(A_i\in \mathbb{R}^{q_i\times p}\). The unobserved signals (or regression coefficients) \(X_i\) are vectors lying on an unknown low-dimensional space.
Given only \(Y_i\) and \(A_i\), to predict or recover \(X_i\) values, the authors develop optimal methods by “borrowing strength” across the different samples. They use linear empirical Bayes methods scale to large datasets and weak moment assumptions.
The model has wide-ranging applications in signal processing, deconvolution, cryo-electron microscopy, and missing data with noise. For missing data, the authors demonstrate in simulations that the proposed methods are more robust to noise and to unequal sampling than well-known matrix completion methods.

MSC:

62H25 Factor analysis and principal components; correspondence analysis
62H15 Hypothesis testing in multivariate analysis
62M20 Inference from stochastic processes and prediction
62D10 Missing data
62P35 Applications of statistics to physics

Software:

SPECTRODE; OptShrink
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Andén, J., Katsevich, E. and Singer, A. (2015). Covariance estimation using conjugate gradient for 3D classification in cryo-EM. In Biomedical Imaging (ISBI), 2015 IEEE 12th International Symposium on 200-204. IEEE, New York.
[2] ASPIRE (2017). Algorithms for single particle reconstruction. Available at http://spr.math.princeton.edu/.
[3] Bai, Z. and Ding, X. (2012). Estimation of spiked eigenvalues in spiked models. Random Matrices Theory Appl. 1 1150011, 21. · Zbl 1251.15037 · doi:10.1142/S2010326311500110
[4] Bai, X.-C., McMullan, G. and Scheres, S. H. (2015). How cryo-EM is revolutionizing structural biology. Trends Biochem. Sci. 40 49-57.
[5] Bai, Z. and Silverstein, J. W. (2009). Spectral Analysis of Large Dimensional Random Matrices. Springer Series in Statistics. Springer, New York. · Zbl 1196.60002
[6] Bai, Z. and Yao, J. (2012). On sample eigenvalues in a generalized spiked population model. J. Multivariate Anal. 106 167-177. · Zbl 1301.62049 · doi:10.1016/j.jmva.2011.10.009
[7] Baik, J., Ben Arous, G. and Péché, S. (2005). Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33 1643-1697. · Zbl 1086.15022 · doi:10.1214/009117905000000233
[8] Baik, J. and Silverstein, J. W. (2006). Eigenvalues of large sample covariance matrices of spiked population models. J. Multivariate Anal. 97 1382-1408. · Zbl 1220.15011 · doi:10.1016/j.jmva.2005.08.003
[9] Benaych-Georges, F. and Nadakuditi, R. R. (2012). The singular values and vectors of low rank perturbations of large rectangular random matrices. J. Multivariate Anal. 111 120-135. · Zbl 1252.15039 · doi:10.1016/j.jmva.2012.04.019
[10] Bhamre, T., Zhang, T. and Singer, A. (2016). Denoising and covariance estimation of single particle cryo-EM images. J. Struct. Biol. 195 72-81.
[11] Blackledge, J. M. (2006). Digital Signal Processing: Mathematical and Computational Methods, Software Development and Applications. Elsevier, Amsterdam. · Zbl 1059.94001
[12] Buja, A. and Eyuboglu, N. (1992). Remarks on parallel analysis. Multivar. Behav. Res. 27 509-540.
[13] Callaway, E. (2015). The revolution will not be crystallized. Nature 525 172.
[14] Campisi, P. and Egiazarian, K., eds. (2016). Blind Image Deconvolution: Theory and Applications. CRC Press, Boca Raton, FL.
[15] Candès, E. J. and Plan, Y. (2010). Matrix completion with noise. Proc. IEEE 98 925-936.
[16] Candès, E. J. and Recht, B. (2009). Exact matrix completion via convex optimization. Found. Comput. Math. 9 717-772. · Zbl 1219.90124 · doi:10.1007/s10208-009-9045-5
[17] Candès, E. J. and Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inform. Theory 56 2053-2080. · Zbl 1366.15021 · doi:10.1109/TIT.2010.2044061
[18] Chen, Y., Bhojanapalli, S., Sanghavi, S. and Ward, R. (2015). Completing any low-rank matrix, provably. J. Mach. Learn. Res. 16 2999-3034. · Zbl 1351.62107
[19] Dobriban, E. (2015). Efficient computation of limit spectra of sample covariance matrices. Random Matrices Theory Appl. 4 1550019, 36. · Zbl 1330.65029
[20] Dobriban, E. (2017). Permutation methods for factor analysis and PCA. Preprint. Available at arXiv:1710.00479. · Zbl 1486.62182 · doi:10.1214/16-AOS1514
[21] Dobriban, E., Leeb, W. and Singer, A. (2019). Supplement to “Optimal prediction in the linearly transformed spiked model.” https://doi.org/10.1214/19-AOS1819SUPP. · Zbl 1441.62158
[22] Dobriban, E. and Owen, A. B. (2019). Deterministic parallel analysis: An improved method for selecting factors and principal components. J. R. Stat. Soc. Ser. B. Stat. Methodol. 81 163-183. · Zbl 1407.62216 · doi:10.1111/rssb.12301
[23] Donoho, D. and Gavish, M. (2014). Minimax risk of matrix denoising by singular value thresholding. Ann. Statist. 42 2413-2440. · Zbl 1310.62014 · doi:10.1214/14-AOS1257
[24] Donoho, D., Gavish, M. and Johnstone, I. (2018). Optimal shrinkage of eigenvalues in the spiked covariance model. Ann. Statist. 46 1742-1778. · Zbl 1403.62099 · doi:10.1214/17-AOS1601
[25] Efron, B. (2012). Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction. Institute of Mathematical Statistics (IMS) Monographs 1. Cambridge Univ. Press, Cambridge. · Zbl 1256.62007
[26] Gavish, M. and Donoho, D. L. (2017). Optimal shrinkage of singular values. IEEE Trans. Inform. Theory 63 2137-2152. · Zbl 1366.94100 · doi:10.1109/TIT.2017.2653801
[27] Golub, G. H. and Van Loan, C. F. (2012). Matrix Computations, Vol. 3. Johns Hopkins Studies in the Mathematical Sciences. Johns Hopkins Univ. Press, Baltimore, MD.
[28] Hachem, W., Hardy, A. and Najim, J. (2015). A survey on the eigenvalues local behavior of large complex correlated Wishart matrices. In Modélisation Aléatoire et Statistique—Journées MAS 2014. ESAIM Proc. Surveys 51 150-174. EDP Sci., Les Ulis. · Zbl 1360.60018 · doi:10.1051/proc/201551009
[29] Jain, P., Netrapalli, P. and Sanghavi, S. (2013). Low-rank matrix completion using alternating minimization. In STOC’13—Proceedings of the 2013 ACM Symposium on Theory of Computing 665-674. ACM, New York. · Zbl 1293.65073
[30] Ji, S. and Ye, J. (2009). An accelerated gradient method for trace norm minimization. In Proceedings of the 26th Annual International Conference on Machine Learning 457-464. ACM, New York.
[31] Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295-327. · Zbl 1016.62078 · doi:10.1214/aos/1009210544
[32] Johnstone, I. M. and Onatski, A. (2015). Testing in high-dimensional spiked models. Preprint. Available at arXiv:1509.07269. · Zbl 1452.62192
[33] Kam, Z. (1980). The reconstruction of structure from electron micrographs of randomly oriented particles. J. Theoret. Biol. 82 15-39.
[34] Katsevich, E., Katsevich, A. and Singer, A. (2015). Covariance matrix estimation for the cryo-EM heterogeneity problem. SIAM J. Imaging Sci. 8 126-185. · Zbl 1362.92036 · doi:10.1137/130935434
[35] Keshavan, R. H. and Montanari, A. (2010). Regularization for matrix completion. In Proceedings of International Symposium on Information Theory 1503-1507. IEEE, New York. · Zbl 1242.62069
[36] Keshavan, R. H., Montanari, A. and Oh, S. (2010). Matrix completion from a few entries. IEEE Trans. Inform. Theory 56 2980-2998. · Zbl 1366.62111 · doi:10.1109/TIT.2010.2046205
[37] Keshavan, R. H., Oh, S. and Montanari, A. (2009). Matrix completion from a few entries. In 2009 IEEE International Symposium on Information Theory 324-328. IEEE, New York. · Zbl 1366.62111 · doi:10.1109/TIT.2010.2046205
[38] Klopp, O. (2014). Noisy low-rank matrix completion with general sampling distribution. Bernoulli 20 282-303. · Zbl 1400.62115 · doi:10.3150/12-BEJ486
[39] Koltchinskii, V., Lounici, K. and Tsybakov, A. B. (2011). Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion. Ann. Statist. 39 2302-2329. · Zbl 1231.62097 · doi:10.1214/11-AOS894
[40] Kritchman, S. and Nadler, B. (2008). Determining the number of components in a factor model from limited noisy data. Chemom. Intell. Lab. Syst. 94 19-32.
[41] Mallat, S. (2008). A Wavelet Tour of Signal Processing: The Sparse Way, with Contributions from Gabriel Peyré. Elsevier/Academic Press, Amsterdam.
[42] Marchenko, V. A. and Pastur, L. A. (1967). Distribution of eigenvalues in certain sets of random matrices. Mat. Sb. 114 507-536. · Zbl 0152.16101
[43] Nadakuditi, R. R. (2014). OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage. IEEE Trans. Inform. Theory 60 3002-3018. · Zbl 1360.62399 · doi:10.1109/TIT.2014.2311661
[44] Nadakuditi, R. R. and Edelman, A. (2008). Sample eigenvalue based detection of high-dimensional signals in white noise using relatively few samples. IEEE Trans. Signal Process. 56 2625-2638. · Zbl 1390.94567 · doi:10.1109/TSP.2008.917356
[45] Nadler, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. Ann. Statist. 36 2791-2817. · Zbl 1168.62058 · doi:10.1214/08-AOS618
[46] Negahban, S. and Wainwright, M. J. (2011). Estimation of (near) low-rank matrices with noise and high-dimensional scaling. Ann. Statist. 39 1069-1097. · Zbl 1216.62090 · doi:10.1214/10-AOS850
[47] Onatski, A. (2012). Asymptotics of the principal components estimator of large factor models with weakly influential factors. J. Econometrics 168 244-258. · Zbl 1443.62497 · doi:10.1016/j.jeconom.2012.01.034
[48] Onatski, A., Moreira, M. J. and Hallin, M. (2013). Asymptotic power of sphericity tests for high-dimensional data. Ann. Statist. 41 1204-1231. · Zbl 1293.62125 · doi:10.1214/13-AOS1100
[49] Onatski, A., Moreira, M. J. and Hallin, M. (2014). Signal detection in high dimension: The multispiked case. Ann. Statist. 42 225-254. · Zbl 1296.62123 · doi:10.1214/13-AOS1181
[50] Passemier, D. and Yao, J.-F. (2012). On determining the number of spikes in a high-dimensional spiked population model. Random Matrices Theory Appl. 1 1150002, 19. · Zbl 1244.62028 · doi:10.1142/S201032631150002X
[51] Paul, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617-1642. · Zbl 1134.62029
[52] Paul, D. and Aue, A. (2014). Random matrix theory in statistics: A review. J. Statist. Plann. Inference 150 1-29. · Zbl 1287.62011 · doi:10.1016/j.jspi.2013.09.005
[53] Recht, B. (2011). A simpler approach to matrix completion. J. Mach. Learn. Res. 12 3413-3430. · Zbl 1280.68141
[54] Rohde, A. and Tsybakov, A. B. (2011). Estimation of high-dimensional low-rank matrices. Ann. Statist. 39 887-930. · Zbl 1215.62056 · doi:10.1214/10-AOS860
[55] Searle, S. R., Casella, G. and McCulloch, C. E. (2009). Variance Components. Wiley Series in Probability and Statistics 391. Wiley Interscience, Hoboken, NJ. · Zbl 1108.62064
[56] Singer, A. and Wu, H.-T. (2013). Two-dimensional tomography from noisy projections taken at unknown random directions. SIAM J. Imaging Sci. 6 136-175. · Zbl 1279.68339 · doi:10.1137/090764657
[57] Srebro, N. and Salakhutdinov, R. R. (2010). Collaborative filtering in a non-uniform world: Learning with the weighted trace norm. In Advances in Neural Information Processing Systems 2056-2064.
[58] Stein, E. M. and Shakarchi, R. (2011). Fourier Analysis: An Introduction. Princeton Lectures in Analysis 1. Princeton Univ. Press, Princeton, NJ. · Zbl 1235.46001
[59] Yao, J., Zheng, S. and Bai, Z. (2015). Large Sample Covariance Matrices and High-Dimensional Data Analysis. Cambridge Series in Statistical and Probabilistic Mathematics 39. Cambridge Univ. Press, New York.
[60] Zhao, Z., Shkolnisky, Y. and Singer, A. (2016). Fast steerable principal component analysis. IEEE Trans. Comput. Imag. 2 1-12.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.