Nonlinear shrinkage estimation of large-dimensional covariance matrices. (English) Zbl 1274.62371

Summary: Many statistical applications require an estimate of a covariance matrix and/or its inverse. When the matrix dimension is large compared to the sample size, which happens frequently, the sample covariance matrix is known to perform poorly and may suffer from ill-conditioning. There already exists an extensive literature concerning improved estimators in such situations. In the absence of further knowledge about the structure of the true covariance matrix, the most successful approach so far, arguably, has been shrinkage estimation. Shrinking the sample covariance matrix to a multiple of the identity, by taking a weighted average of the two, turns out to be equivalent to linearly shrinking the sample eigenvalues to their grand mean, while retaining the sample eigenvectors. Our paper extends this approach by considering nonlinear transformations of the sample eigenvalues. We show how to construct an estimator that is asymptotically equivalent to an oracle estimator suggested in previous work. As demonstrated in extensive Monte Carlo simulations, the resulting bona fide estimator can result in sizeable improvements over the sample covariance matrix and also over linear shrinkage.


62H12 Estimation in multivariate analysis
62G20 Asymptotic properties of nonparametric inference
60B20 Random matrices (probabilistic aspects)


Full Text: DOI arXiv Euclid


[1] Bai, Z. D. and Silverstein, J. W. (1998). No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. Ann. Probab. 26 316-345. · Zbl 0937.60017
[2] Bickel, P. J. and Levina, E. (2008). Regularized estimation of large covariance matrices. Ann. Statist. 36 199-227. · Zbl 1132.62040
[3] Cai, T. and Zhou, H. (2012). Minimax estimation of large covariance matrices under \(\ell _1\) norm. Statist. Sinica . · Zbl 1266.62036
[4] El Karoui, N. (2008). Spectrum estimation for large dimensional covariance matrices using random matrix theory. Ann. Statist. 36 2757-2790. · Zbl 1168.62052
[5] Fan, J., Fan, Y. and Lv, J. (2008). High dimensional covariance matrix estimation using a factor model. J. Econometrics 147 186-197. · Zbl 1429.62185
[6] Gill, P. E., Murray, W. and Saunders, M. A. (2002). SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM J. Optim. 12 979-1006 (electronic). · Zbl 1027.90111
[7] Haff, L. R. (1980). Empirical Bayes estimation of the multivariate normal covariance matrix. Ann. Statist. 8 586-597. · Zbl 0441.62045
[8] James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4 th Berkeley Sympos. Math. Statist. and Prob. , Vol. I 361-379. Univ. California Press, Berkeley, Calif. · Zbl 1281.62026
[9] Khare, K. and Rajaratnam, B. (2011). Wishart distributions for decomposable covariance graph models. Ann. Statist. 39 514-555. · Zbl 1274.62369
[10] Ledoit, O. and Péché, S. (2011). Eigenvectors of some large sample covariance matrix ensembles. Probab. Theory Related Fields 151 233-264. · Zbl 1229.60009
[11] Ledoit, O. and Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365-411. · Zbl 1032.62050
[12] Ledoit, O. and Wolf, M. (2012). Supplement to “Nonlinear shrinkage estimation of large-dimensional covariance matrices.” . · Zbl 1274.62371
[13] Marčenko, V. A. and Pastur, L. A. (1967). Distribution of eigenvalues for some sets of random matrices. Sbornik : Mathematics 1 457-483. · Zbl 0162.22501
[14] Mestre, X. and Lagunas, M. A. (2006). Finite sample size effect on minimum variance beamformers: Optimum diagonal loading factor for large arrays. IEEE Trans. Signal Process. 54 69-82. · Zbl 1373.94666
[15] Perlman, M. D. (2007). STAT 542: Multivariate Statistical Analysis . Univ. Washington (On-Line Class Notes), Seattle, Washington.
[16] Rajaratnam, B., Massam, H. and Carvalho, C. M. (2008). Flexible covariance estimation in graphical Gaussian models. Ann. Statist. 36 2818-2849. · Zbl 1168.62054
[17] Ravikumar, P., Wawinwright, M., Raskutti, G. and Yu, B. (2008). High-dimensional covariance estimation by minimizing \(\ell_1\)-penalized log-determinant divergence Technical Report 797, Dept. Statistics, Univ. California, Berkeley.
[18] Rohde, A. and Tsybakov, A. B. (2011). Estimation of high-dimensional low-rank matrices. Ann. Statist. 39 887-930. · Zbl 1215.62056
[19] Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis . Chapman & Hall, London. · Zbl 0617.62042
[20] Silverstein, J. W. (1995). Strong convergence of the empirical distribution of eigenvalues of large-dimensional random matrices. J. Multivariate Anal. 55 331-339. · Zbl 0851.62015
[21] Silverstein, J. W. and Choi, S.-I. (1995). Analysis of the limiting spectral distribution of large-dimensional random matrices. J. Multivariate Anal. 54 295-309. · Zbl 0872.60013
[22] Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability , 1954 - 1955, Vol. I 197-206. Univ. California Press, Berkeley. · Zbl 0073.35602
[23] Stein, C. (1975). Estimation of a covariance matrix. Rietz lecture, 39th Annual Meeting IMS. Atlanta, Georgia. · Zbl 0302.05015
[24] Won, J. H., Lim, J., Kim, S. J. and Rajaratnam, B. (2009). Maximum likelihood covariance estimation with a condition number constraint. Technical Report 2009-10, Dept. Statistics, Stanford Univ.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.