## Estimation of smooth functionals in normal models: bias reduction and asymptotic efficiency.(English)Zbl 1486.62163

Summary: Let $${X_1},\dots ,{X_n}$$ be i.i.d. random variables sampled from a normal distribution $$N(\mu ,\Sigma )$$ in $${\mathbb{R}^d}$$ with unknown parameter $$\theta =(\mu ,\Sigma )\in \Theta :={\mathbb{R}^d}\times{\mathcal{C}_+^d}$$, where $${\mathcal{C}_+^d}$$ is the cone of positively definite covariance operators in $${\mathbb{R}^d}$$. Given a smooth functional $$f:\Theta \mapsto{\mathbb{R}^1}$$, the goal is to estimate $$f(\theta )$$ based on $${X_1},\dots ,{X_n}$$. Let $\Theta (a;d):={\mathbb{R}^d}\times{\Sigma \in{\mathcal{C}_+^d}:\sigma (\Sigma )\subset [1/a,a]}, \quad a\ge 1,$ where $$\sigma (\Sigma )$$ is the spectrum of covariance $$\Sigma$$. Let $$\hat{\theta }:=(\hat{\mu },\hat{\Sigma })$$, where $$\hat{\mu }$$ is the sample mean and $$\hat{\Sigma }$$ is the sample covariance, based on the observations $${X_1},\dots ,{X_n}$$. For an arbitrary functional $$f\in{C^s}(\Theta )$$, $$s=k+1+\rho$$, $$k\ge 0$$, $$\rho \in (0,1]$$, we define a functional $${f_k}:\Theta \mapsto \mathbb{R}$$ such that $\underset{\theta \in \Theta (a;d)}{\sup }{\| {f_k}(\hat{\theta })-f(\theta )\|_{{L_2}({\mathbb{P}_{\theta }})}} \lesssim_{s,\beta } \|f\|_{C^s(\Theta)} \left[ \left( \frac{a}{\sqrt{n}} \vee {a^{\beta s}} \left( \sqrt{\frac{d}{n}}\right)^s \right) \wedge 1 \right],$ where $$\beta =1$$ for $$k=0$$ and $$\beta > s-1$$ is arbitrary for $$k\ge 1$$. This error rate is minimax optimal and similar bounds hold for more general loss functions. If $$d={d_n}\le{n^{\alpha }}$$ for some $$\alpha \in (0,1)$$ and $$s\ge \frac{1}{1-\alpha }$$, the rate becomes $$O({n^{-1/2}})$$. Moreover, for $$s > \frac{1}{1-\alpha }$$, the estimator $${f_k}(\hat{\theta })$$ is shown to be asymptotically efficient. The crucial part of the construction of estimator $${f_k}(\hat{\theta })$$ is a bias reduction method studied in the paper for more general statistical models than normal.

### MSC:

 62H12 Estimation in multivariate analysis 60B20 Random matrices (probabilistic aspects) 62G20 Asymptotic properties of nonparametric inference 62H25 Factor analysis and principal components; correspondence analysis
Full Text:

### References:

 [1] Adamczak, R. and Wolff, P. (2015). Concentration inequalities for non-Lipschitz functions with bounded derivatives of higher order. Probab. Theory Related Fields 162 531-586. · Zbl 1323.60033 · doi:10.1007/s00440-014-0579-3 [2] Aleksandrov, A. B. and Peller, V. V. (2016). Operator Lipschitz functions. Uspekhi Mat. Nauk 71 3-106. · Zbl 1356.26002 · doi:10.4213/rm9729 [3] Bickel, P. J., Klaassen, C. A. J., Ritov, Y. and Wellner, J. A. (1993). Efficient and Adaptive Estimation for Semiparametric Models. Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins Univ. Press, Baltimore, MD. · Zbl 0786.62001 [4] Bickel, P. J. and Ritov, Y. (1988). Estimating integrated squared density derivatives: Sharp best order of convergence estimates. Sankhyā Ser. A 50 381-393. · Zbl 0676.62037 [5] Birgé, L. and Massart, P. (1995). Estimation of integral functionals of a density. Ann. Statist. 23 11-29. · Zbl 0848.62022 · doi:10.1214/aos/1176324452 [6] Cai, T. T. and Low, M. G. (2005). On adaptive estimation of linear functionals. Ann. Statist. 33 2311-2343. · Zbl 1086.62031 · doi:10.1214/009053605000000633 [7] Cai, T. T. and Low, M. G. (2005). Nonquadratic estimators of a quadratic functional. Ann. Statist. 33 2930-2956. · Zbl 1085.62055 · doi:10.1214/009053605000000147 [8] Collier, O., Comminges, L. and Tsybakov, A. B. (2017). Minimax estimation of linear and quadratic functionals on sparsity classes. Ann. Statist. 45 923-958. · Zbl 1368.62191 · doi:10.1214/15-AOS1432 [9] Donoho, D. and Liu, R. (1987). On minimax estimation of linear functionals Technical Report N 105, Department of Statistics, UC Berkeley. [10] Donoho, D. L. and Liu, R. C. (1991). Geometrizing rates of convergence. II, III. Ann. Statist. 19 633-667, 668-701. · Zbl 0754.62028 · doi:10.1214/aos/1176348114 [11] Donoho, D. L. and Nussbaum, M. (1990). Minimax quadratic estimation of a quadratic functional. J. Complexity 6 290-323. · Zbl 0724.62039 · doi:10.1016/0885-064X(90)90025-9 [12] Faà di Bruno, F. (1855). Sullo sviluppo delle funzioni. Annali di Scienze Matematiche e Fisiche 6 479-480. [13] Gill, R. D. and Levit, B. Y. (1995). Applications of the Van Trees inequality: A Bayesian Cramér-Rao bound. Bernoulli 1 59-79. · Zbl 0830.62035 · doi:10.2307/3318681 [14] Girko, V. L. (1987). An introduction to general statistical analysis. Theory Probab. Appl. 32 229-242. · Zbl 0653.62043 [15] Girko, V. L. (1995). Statistical Analysis of Observations of Increasing Dimension. Theory and Decision Library. Series B: Mathematical and Statistical Methods 28. Kluwer Academic, Dordrecht. · Zbl 0867.62047 · doi:10.1007/978-94-015-8567-5 [16] Han, Y., Jiao, J. and Mukherjee, R. (2020). On estimation of ${L_r}$-norms in Gaussian white noise models. Probab. Theory Related Fields 177 1243-1294. · Zbl 1445.62072 · doi:10.1007/s00440-020-00982-x [17] Hardy, M. (2006). Combinatorics of partial derivatives. Electron. J. Combin. 13 Research Paper 1. · Zbl 1080.05006 [18] Ibragimov, I. A. and Has’minskiĭ, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Applications of Mathematics 16. Springer, New York. [19] Ibragimov, I. A., Nemirovskiĭ, A. S. and Khas’minskiĭ, R. Z. (1987). Some problems of nonparametric estimation in Gaussian white noise. Theory Probab. Appl. 31 391-406. · Zbl 0623.62028 [20] Jiao, J. and Han, Y. (2020). Bias correction with jackknife, bootstrap, and Taylor series. IEEE Trans. Inf. Theory 66 4392-4418. · Zbl 1446.62104 · doi:10.1109/TIT.2020.2969439 [21] Jost, J., Kell, M. and Rodrigues, C. S. (2015). Representation of Markov chains by random maps: Existence and regularity conditions. Calc. Var. Partial Differential Equations 54 2637-2655. · Zbl 1352.37064 · doi:10.1007/s00526-015-0878-2 [22] Klemelä, J. (2006). Sharp adaptive estimation of quadratic functionals. Probab. Theory Related Fields 134 539-564. · Zbl 1082.62032 · doi:10.1007/s00440-005-0447-2 [23] Koltchinskii, V. (2020). Estimation of smooth functionals in high-dimensional models: bootstrap chains and Gaussian approximation. Available at arXiv:2011.03789. [24] Koltchinskii, V. (2018). Asymptotic efficiency in high-dimensional covariance estimation. In Proceedings of the International Congress of Mathematicians—Rio de Janeiro 2018. Vol. IV. Invited Lectures 2921-2943. World Sci. Publ., Hackensack, NJ. · Zbl 1451.62057 [25] Koltchinskii, V. (2021). Asymptotically efficient estimation of smooth functionals of covariance operators. J. Eur. Math. Soc. (JEMS). 23 765-843. · Zbl 1461.62068 · doi:10.4171/JEMS/1023 [26] Koltchinskii, V., Löffler, M. and Nickl, R. (2020). Efficient estimation of linear functionals of principal components. Ann. Statist. 48 464-490. · Zbl 1440.62232 · doi:10.1214/19-AOS1816 [27] Koltchinskii, V. and Lounici, K. (2016). Asymptotics and concentration bounds for bilinear forms of spectral projectors of sample covariance. Ann. Inst. Henri Poincaré Probab. Stat. 52 1976-2013. · Zbl 1353.62053 · doi:10.1214/15-AIHP705 [28] Koltchinskii, V. and Lounici, K. (2017). Concentration inequalities and moment bounds for sample covariance operators. Bernoulli 23 110-133. · Zbl 1366.60057 · doi:10.3150/15-BEJ730 [29] Koltchinskii, V. and Lounici, K. (2017). Normal approximation and concentration of spectral projectors of sample covariance. Ann. Statist. 45 121-157. · Zbl 1367.62175 · doi:10.1214/16-AOS1437 [30] Koltchinskii, V. and Xia, D. (2016). Perturbation of linear forms of singular vectors under Gaussian noise. In High Dimensional Probability VII. Progress in Probability 71 397-423. Springer, Cham. · Zbl 1353.15034 · doi:10.1007/978-3-319-40519-3_18 [31] Koltchinskii, V. and Zhilova, M. (2021). Efficient estimation of smooth functionals in Gaussian shift models. Ann. Inst. Henri Poincaré Probab. Stat. To appear. Available at arXiv:1810.02767. · Zbl 1469.62430 [32] Koltchinskii, V. and Zhilova, M. (2021). Estimation of smooth functionals of location parameter in Gaussian and Poincaré random shift models. Sankhyā. To appear. · Zbl 1479.62035 [33] Koltchinskii, V. and Zhilova, M. (2021). Supplement to “Estimation of smooth functionals in normal models: Bias reduction and asymptotic efficiency.” https://doi.org/10.1214/20-AOS2047SUPP. · Zbl 1469.62430 [34] Laurent, B. (1996). Efficient estimation of integral functionals of a density. Ann. Statist. 24 659-681. · Zbl 0859.62038 · doi:10.1214/aos/1032894458 [35] Lepski, O., Nemirovski, A. and Spokoiny, V. (1999). On estimation of the ${L_r}$ norm of a regression function. Probab. Theory Related Fields 113 221-253. · Zbl 0921.62103 · doi:10.1007/s004409970006 [36] Levit, B. Y. (1975). The efficiency of a certain class of nonparametric estimators. Theory Probab. Appl. 20 723-740. · Zbl 0367.62041 [37] Levit, B. Y. (1978). Asymptotically efficient estimation of nonlinear functionals. Problemy Peredachi Informatsii 14 65-72. · Zbl 0422.62034 [38] Mukherjee, R., Newey, W. and Robins, J. (2017). Semiparametric efficient empirical higher order influence function estimators. Available at arXiv:1705.07577. [39] Nemirovski, A. (2000). Topics in non-parametric statistics. In Lectures on Probability Theory and Statistics (Saint-Flour, 1998). Lecture Notes in Math. 1738 85-277. Springer, Berlin. · Zbl 0998.62033 [40] Nemirovskiĭ, A. S. (1990). Necessary conditions for efficient estimation of functionals of a nonparametric signal observed in white noise. Theory Probab. Appl. 35 94-103. · Zbl 0721.62043 [41] Pisier, G. (1986). Probabilistic methods in the geometry of Banach spaces. In Probability and Analysis (Varenna, 1985). Lecture Notes in Math. 1206 167-241. Springer, Berlin. · Zbl 0606.60008 · doi:10.1007/BFb0076302 [42] Robins, J., Li, L., Tchetgen, E. and van der Vaart, A. (2008). Higher order influence functions and minimax estimation of nonlinear functionals. In Probability and Statistics: Essays in Honor of David A. Freedman. Inst. Math. Stat. (IMS) Collect. 2 335-421. IMS, Beachwood, OH. · Zbl 1513.62067 · doi:10.1214/193940307000000527 [43] Robins, J. M., Li, L., Tchetgen, E. T. and van der Vaart, A. (2016). Asymptotic normality of quadratic estimators. Stochastic Process. Appl. 126 3733-3759. · Zbl 1348.62170 · doi:10.1016/j.spa.2016.04.005 [44] Triebel, H. (1983). Theory of Function Spaces. Monographs in Mathematics 78. Birkhäuser, Basel. · Zbl 1235.46002 · doi:10.1007/978-3-0346-0416-1 [45] van der Vaart, A. (2014). Higher order tangent spaces and influence functions. Statist. Sci. 29 679-686. · Zbl 1331.62111 · doi:10.1214/14-STS478 [46] Villani, C. (2009). Optimal Transport. Old and New. Springer, Berlin · Zbl 1156.53003
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.