×

Confidence sets in sparse regression. (English) Zbl 1288.62108

Summary: The problem of constructing confidence sets in a high-dimensional linear model with \(n\) response variables and \(p\) parameters, possibly \(p \geq n\), is considered. Full honest adaptive inference is possible if the rate of sparse estimation does not exceed \(n^{-1/4}\), otherwise sparse adaptive confidence sets exist only over strict subsets of the parameter spaces for which sparse estimators exist. Necessary and sufficient conditions for the existence of confidence sets that adapt to a fixed sparsity level of the parameter vector are given in terms of minimal \(\ell^{2}\)-separation conditions on the parameter space. The design conditions cover common coherence assumptions used in models for sparsity, including (possibly correlated) sub-Gaussian designs.

MSC:

62J05 Linear regression; mixed models
62F25 Parametric tolerance and confidence regions
62G15 Nonparametric tolerance and confidence regions
PDF BibTeX XML Cite
Full Text: DOI arXiv Euclid

References:

[1] Alexander, K. S. (1985). Rates of growth for weighted empirical processes. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer , Vol. II ( Berkeley , Calif. , 1983). 2 475-493. Wadsworth, Belmont, CA. · Zbl 1372.60051
[2] Arias-Castro, E., Candès, E. J. and Plan, Y. (2011). Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism. Ann. Statist. 39 2533-2556. · Zbl 1231.62136
[3] Baraud, Y. (2004). Confidence balls in Gaussian regression. Ann. Statist. 32 528-551. · Zbl 1093.62051
[4] Beran, R. and Dümbgen, L. (1998). Modulation of estimators and confidence sets. Ann. Statist. 26 1826-1856. · Zbl 1073.62538
[5] Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of lasso and Dantzig selector. Ann. Statist. 37 1705-1732. · Zbl 1173.62022
[6] Boucheron, S., Lugosi, G. and Massart, P. (2013). Concentration Inequalities. A Nonasymptotic Theory of Independence . Oxford Univ. Press, London. · Zbl 1279.60005
[7] Bühlmann, P. and van de Geer, S. (2011). Statistics for High-Dimensional Data : Methods , Theory and Applications . Springer, Heidelberg. · Zbl 1273.62015
[8] Bull, A. D. and Nickl, R. (2013). Adaptive confidence sets in \(L^{2}\). Probab. Theory Related Fields 156 889-919. · Zbl 1273.62105
[9] Cai, T. T. and Low, M. G. (2006). Adaptive confidence balls. Ann. Statist. 34 202-228. · Zbl 1091.62037
[10] Candès, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when \(p\) is much larger than \(n\). Ann. Statist. 35 2313-2351. · Zbl 1139.62019
[11] Dudley, R. M. (1967). The sizes of compact subsets of Hilbert space and continuity of Gaussian processes. J. Funct. Anal. 1 290-330. · Zbl 0188.20502
[12] Giné, E. and Nickl, R. (2009). An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive estimation. Probab. Theory Related Fields 143 569-596. · Zbl 1160.62032
[13] Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122-1170. · Zbl 1183.62062
[14] Hoffmann, M. and Lepski, O. (2002). Random rates in anisotropic regression. Ann. Statist. 30 325-396. · Zbl 1012.62042
[15] Hoffmann, M. and Nickl, R. (2011). On adaptive inference and confidence bands. Ann. Statist. 39 2383-2409. · Zbl 1232.62072
[16] Ingster, Y. I., Tsybakov, A. B. and Verzelen, N. (2010). Detection boundary in sparse regression. Electron. J. Stat. 4 1476-1526. · Zbl 1329.62314
[17] Javanmard, A. and Montanari, A. (2013). Confidence intervals and hypothesis testing for high-dimensional regression. Available at . 1306.3171 · Zbl 1319.62145
[18] Juditsky, A. and Lambert-Lacroix, S. (2003). Nonparametric confidence set estimation. Math. Methods Statist. 12 410-428.
[19] Li, K.-C. (1989). Honest confidence regions for nonparametric regression. Ann. Statist. 17 1001-1008. · Zbl 0681.62047
[20] Loh, P.-L. and Wainwright, M. J. (2012). High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity. Ann. Statist. 40 1637-1664. · Zbl 1257.62063
[21] Pötscher, B. M. (2009). Confidence sets based on sparse estimators are necessarily large. Sankhyā 71 1-18. · Zbl 1192.62096
[22] Pötscher, B. M. and Schneider, U. (2011). Distributional results for thresholding estimators in high-dimensional Gaussian regression models. Electron. J. Stat. 5 1876-1934. · Zbl 1271.62149
[23] Robins, J. and van der Vaart, A. (2006). Adaptive nonparametric confidence sets. Ann. Statist. 34 229-253. · Zbl 1091.62039
[24] Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505-563. · Zbl 0893.60001
[25] van de Geer, S. A. (2000). Empirical Processes in M-Estimation . Cambridge Univ. Press, Cambridge. · Zbl 0953.62049
[26] van de Geer, S. (2001). Least squares estimation with complexity penalties. Math. Methods Statist. 10 355-374. · Zbl 1005.62043
[27] van de Geer, S., Bühlmann, P. and Ritov, Y. (2013). On asymptotically optimal confidence regions and tests for high-dimensional models. Submitted. Available at . 1303.0518 · Zbl 1305.62259
[28] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer Series in Statistics . Springer, New York. · Zbl 0862.60002
[29] Zhang, C. H. and Zhang, S. S. (2011). Confidence intervals for low-dimensional parameters with high-dimensional data. 2011. Available at . 1110.2563v1
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.