Spectral analysis of high-dimensional sample covariance matrices with missing observations. (English) Zbl 1388.62152

Jurczak, Kamil; Rohde, Angelika

Bernoulli 23, No. 4A, 2466-2532 (2017).

Summary: We study high-dimensional sample covariance matrices based on independent random vectors with missing coordinates. The presence of missing observations is common in modern applications such as climate studies or gene expression micro-arrays. A weak approximation on the spectral distribution in the “large dimension \(d\) and large sample size \(n\)” asymptotics is derived for possibly different observation probabilities in the coordinates. The spectral distribution turns out to be strongly influenced by the missingness mechanism. In the null case under the missing at random scenario where each component is observed with the same probability \(p\), the limiting spectral distribution is a Marčenko-Pastur law [V. A. Marchenko and L. A. Pastur, Math. USSR, Sb. 1, 457–483 (1968; Zbl 0162.22501); Mat. Sb., Nov. Ser. 72(114), 507–536 (1967; Zbl 0152.16101)] shifted by \((1-p)/p\) to the left. As \(d/n\rightarrow y\in(0,1)\), the almost sure convergence of the extremal eigenvalues to the respective boundary points of the support of the limiting spectral distribution is proved, which are explicitly given in terms of \(y\) and \(p\). Eventually, the sample covariance matrix is positive definite if \(p\) is larger than \[ 1-(1-\sqrt{y})^{2}, \] whereas this is not true any longer if \(p\) is smaller than this quantity.

MSC:

62H05	Characterization and structure theory for multivariate probability distributions; copulas
60B10	Convergence of probability measures
60B20	Random matrices (probabilistic aspects)
62P12	Applications of statistics to environmental and related topics

Keywords:

almost sure convergence of extremal eigenvalues; characterization of positive definiteness; limiting spectral distribution; sample covariance matrix with missing observations; Stieltjes transform; Marčenko-Pastur law

Citations:

Zbl 0162.22501; Zbl 0152.16101