zbMATH — the first resource for mathematics

Sub-Gaussian estimators of the mean of a random vector. (English) Zbl 1417.62192
The mean \(\mu\) of a random vector \(X\) distributed in \(\mathbb{R}^d\) is estimated based on an i.i.d. sample of \(N\) points. If \(X\) has a multivariate normal distribution, then the sample mean \(\overline{\mu}_N\) is also multivariate normal, hence with probability at least \(1-\delta,\) \[ \|\overline{\mu}_N - \mu\|\le \sqrt{\frac{\mathrm{Tr}(\Sigma)}{N}}+\sqrt{\frac{2\lambda_{\max}\log(\frac{1}{\delta})}{N}}, \] where \(\Sigma\) is the covariance matrix and \(\lambda_{\max}\) stands for its largest eigenvalue; see [D. L. Hanson and F. T. Wright, Ann. Math. Stat. 42, 1079–1083 (1971; Zbl 0216.22203)]. Similar bounds hold true if \(X\) has a sub-Gaussian distribution.
In the paper under review, a new mean estimator \(\hat{\mu}_N^{(\delta)}\) is constructed that achieves purely sub-Gaussian performance under the minimal condition that the second moments of \(X\) are finite. Namely, for all \(N,\) with probability at least \(1-\delta,\) it holds \[ \|\hat{\mu}_N^{(\delta)} - \mu\| \le C\left(\sqrt{\frac{\mathrm{Tr}(\Sigma)}{N}}+\sqrt{\frac{\lambda_{\max}\log(\frac{2}{\delta})}{N}}\right) \] for an explicit universal constant \(C.\) Since the bound does not depend on the dimension \(d\) explicitly, the same estimator may be defined for Hilbert-space valued random vectors and the bound remains valid as long as the strong second moment of \(X\) is finite.

62J05 Linear regression; mixed models
62G08 Nonparametric regression and quantile regression
62G35 Nonparametric robustness
62H12 Estimation in multivariate analysis
Full Text: DOI Euclid
[1] Alon, N., Matias, Y. and Szegedy, M. (1999). The space complexity of approximating the frequency moments. J. Comput. System Sci.58 137–147. · Zbl 0938.68153
[2] Aloupis, G. (2006). Geometric measures of data depth. In Data Depth: Robust Multivariate Analysis, Computational Geometry and Applications. DIMACS Ser. Discrete Math. Theoret. Comput. Sci.72 147–158. Amer. Math. Soc., Providence, RI.
[3] Boucheron, S., Lugosi, G. and Massart, P. (2013). Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford Univ. Press, Oxford. · Zbl 1279.60005
[4] Catoni, O. (2012). Challenging the empirical mean and empirical variance: A deviation study. Ann. Inst. Henri Poincaré Probab. Stat.48 1148–1185. · Zbl 1282.62070
[5] Cohen, M. B., Lee, Y. T., Miller, G., Pachocki, J. and Sidford, A. (2016). Geometric median in nearly linear time. In STOC’16—Proceedings of the 48th Annual ACM SIGACT Symposium on Theory of Computing 9–21. ACM, New York. · Zbl 1377.68267
[6] Devroye, L., Lerasle, M., Lugosi, G. and Oliveira, R. I. (2016). Sub-Gausssian mean estimators. Ann. Statist.44 2695–2725. · Zbl 1360.62115
[7] Hanson, D. L. and Wright, F. T. (1971). A bound on tail probabilities for quadratic forms in independent random variables. Ann. Math. Stat.42 1079–1083. · Zbl 0216.22203
[8] Hsu, D. (2010). Robust statistics. Available at http://www.inherentuncertainty.org/2010/12/robust-statistics.html.
[9] Hsu, D. and Sabato, S. (2016). Loss minimization and parameter estimation with heavy tails. J. Mach. Learn. Res.17 Paper No. 18. · Zbl 1360.62380
[10] James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I 361–379. Univ. California Press, Berkeley, CA. · Zbl 1281.62026
[11] Jerrum, M. R., Valiant, L. G. and Vazirani, V. V. (1986). Random generation of combinatorial structures from a uniform distribution. Theoret. Comput. Sci.43 169–188. · Zbl 0597.68056
[12] Joly, E., Lugosi, G. and Oliveira, R. I. (2016). On the estimation of the mean of a random vector. Preprint. · Zbl 1362.62121
[13] Ledoux, M. and Talagrand, M. (1991). Probability in Banach Space. Springer, New York. · Zbl 0748.60004
[14] Lerasle, M. and Oliveira, R. I. (2012). Robust empirical mean estimators. Available at arXiv:1112.3914.
[15] Minsker, S. (2015). Geometric median and robust estimation in Banach spaces. Bernoulli21 2308–2335. · Zbl 1348.60041
[16] van der Waart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York. · Zbl 0862.60002
[17] Vershynin, R. (2009). Lectures in geometric functional analysis. Available at https://www.math.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.