×

zbMATH — the first resource for mathematics

Variable selection via adaptive false negative control in linear regression. (English) Zbl 1434.62156
Summary: Variable selection methods have been developed in linear regression to provide sparse solutions. Recent studies have focused on further interpretations on the sparse solutions in terms of false positive control. In this paper, we consider false negative control for variable selection with the goal to efficiently select a high proportion of relevant predictors. Different from existing studies in power analysis and sure screening, we propose to directly estimate the false negative proportion (FNP) of a decision rule and select the smallest subset of predictors that has the estimated FNP less than a user-specified control level. The proposed method is adaptive to the user-specified control level on FNP by selecting less candidates if a higher level is implemented. On the other hand, when data has stronger effect size or larger sample size, the proposed method controls FNP more efficiently with less false positives. New analytic techniques are developed to cope with the major challenge of FNP control when relevant predictors cannot be consistently separated from irrelevant ones. Our numerical results are in line with the theoretical findings.
MSC:
62J07 Ridge regression; shrinkage estimators (Lasso)
62F03 Parametric hypothesis testing
PDF BibTeX XML Cite
Full Text: DOI Euclid arXiv
References:
[1] Arias-Castro, E., E. J. Cand’s, and Y. Plan (2011). Global testing under sparse alternatives: Anova, multiple comparisons and the higher criticism., Ann. Statist. 39(5), 2533-2556. · Zbl 1231.62136
[2] Arias-Castro, E. and A. Ying (2019). Detection of sparse mixtures: Higher criticism and scan statistic., Electron. J. Statist. 13(1), 208-230. · Zbl 1411.62161
[3] Barber, R. F. and E. J. Candès (2015). Controlling the false discovery rate via knockoffs., Ann. Statist. 43(5), 2055-2085. · Zbl 1327.62082
[4] Bogdan, M., E. Van Den Berg, C. Sabatti, W. Su, and E. J. Candès (2015). Slope—adaptive variable selection via convex optimization., Ann. Appl. Stat. 9(3), 1103-1140. · Zbl 1454.62212
[5] Bühlmann, P. and S. van de Geer (2011)., Statistics for High-Dimensional Data Methods, Theory and Applications. Springer.
[6] Cai, T. T. and Z. Guo (2017). Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity., Ann. Statist. 45(2), 615-646. · Zbl 1371.62045
[7] Cai, T. T., X. J. Jeng, and J. Jin (2011). Optimal detection of heterogeneous and heteroscedastic mixtures., J. R. Statist. Soc. Ser. B 73(5), 629-662. · Zbl 1228.62020
[8] Cai, T. T., W. Liu, and H. H. Zhou (2016, 04). Estimating sparse precision matrix: Optimal rates of convergence and adaptive estimation., Ann. Statist. 44(2), 455-488. · Zbl 1341.62115
[9] Candes, E., Y. Fan, L. Janson, and J. Lv (2018). Panning for gold: Model-x knockoffs for high dimensional controlled variable selection., J. R. Statist. Soc. Ser. B 80(3), 551-577. · Zbl 1398.62335
[10] Chen, X. and R. Doerge (2016). A strong law of large numbers related to multiple testing normal means., arXiv:1410.4276v3.
[11] Dezeure, R., P. Bühlmann, L. Meier, and N. Meinshausen (2015). High-dimensional inference: Confidence intervals, p-values and r-software hdi., Statist. Sci. 30(4), 533-558. · Zbl 1426.62183
[12] Donoho, D. and J. Jin (2004). Higher criticism for detecting sparse heterogeneous mixtures., Ann. Statist., 962-994. · Zbl 1092.62051
[13] Fan, J., X. Han, and W. Gu (2012). Estimating false discovery proportion under arbitrary covariance dependence., J. Amer. Statist. Assoc. 107(499), 1019-1035. · Zbl 1395.62219
[14] Fan, J. and J. Lv (2008). Sure independence screening for ultrahigh dimensional feature space., J. R. Statist. Soc. Ser. B 70(5), 849-911. · Zbl 1411.62187
[15] Fan, J. and J. Lv (2010). A selective overview of variable selection in high dimensional feature space., Statistica Sinica 20(1), 101-148. · Zbl 1180.62080
[16] Feller, W. (1971)., An Introduction to Probability Theory and its Applications, Volume II. Wiley, NewYork, NY. · Zbl 0219.60003
[17] G’sell, M., S. Wager, A. Chouldechova, and R. Tibshirani (2016). Sequential selection procedures and false discovery rate control., J. R. Statist. Soc. Ser. B 78(2), 423-444. · Zbl 1414.62341
[18] Hall, P. and J. Jin (2008). Properties of higher criticism under strong dependence., Ann. Statist. 36(1), 381-402. · Zbl 1139.62049
[19] Javanmard, A. and A. Montanari (2013). Model selection for high-dimensional regression under the generalized irrepresentability condition. In, Advances in neural information processing systems, pp. 3012-3020.
[20] Javanmard, A. and A. Montanari (2014). Confidence intervals and hypothesis testing for high-dimensional regression., J. Mach. Learn. Res. 15(1), 2869-2909. · Zbl 1319.62145
[21] Javanmard, A. and A. Montanari (2018). Debiasing the lasso: Optimal sample size for gaussian designs., Ann. Statist. 46(6A), 2593-2622. · Zbl 1407.62270
[22] Jeng, X. J. and X. Chen (2019). Predictor ranking and false discovery proportion control in high-dimensional regression., J. Multivariate Anal. 171, 163-175. · Zbl 1417.62200
[23] Ji, P. and J. Jin (2012). Ups delivers optimal phase diagram in high-dimensional variable selection., Ann. Statist. 40(1), 73-103. · Zbl 1246.62160
[24] Ji, P. and Z. Zhao (2014). Rate optimal multiple testing procedure in high-dimensional regression., arXiv:1404.2961.
[25] Liu, Z., V. J. Berrocal, A. J. Bartsch, and T. D. Johnson (2016). Pre-surgical fmri data analysis using a spatially adaptive conditionally autoregressive model., Bayesian analysis (Online) 11(2), 599. · Zbl 1359.62407
[26] Loring, D., K. Meador, J. D. Allison, J. Pillai, T. Lavin, G. P. Lee, A. Balan, and V. Dave (2002). Now you see it, now you don’t: statistical and methodological considerations in fmri., Epilepsy & Behavior 3(6), 539-547.
[27] Mehler, G. F. (1866). Ueber die entwicklung einer funktion von beliebeg vielen variablen nach laplaceschen funktionen hoherer ordnung., J. Reine Angew. Math. 66, 161-176.
[28] Meinshausen, N. and P. Bühlmann (2006). High-dimensional graphs and variable selection with the lasso., Ann. Statist. 34(3), 1436-1462. · Zbl 1113.62082
[29] Meinshausen, N. and J. Rice (2006). Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses., Ann. Statist. 34(1), 373-393. · Zbl 1091.62059
[30] Powers, D. (2011). Evaluation: From precision, recall and f-measure to roc, informedness, markedness and correlation., J. of Machine Learning Technologies 2(1), 37-63.
[31] Silva, M., A. See, W. Essayed, A. Golby, and Y. Tie (2018). Challenges and techniques for presurgical brain mapping with functional mri., NeuroImage: Clinical 17, 794-803.
[32] Su, W. and E. Candés (2016). Slope is adaptive to unknown sparsity and asymptotically minimax., Ann. Statist. 44, 1038-1068. · Zbl 1338.62032
[33] Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso., J. R. Statist. Soc. Ser. B 58(1), 267-288. · Zbl 0850.62538
[34] van de Geer, S., P. Bühlmann, Y. Ritov, and R. Dezeure (2014). On asymptotically optimal confidence regions and tests for high-dimensional models., Ann. Statist. 42(3), 1166-1202. · Zbl 1305.62259
[35] Wainwright, M. J. (2009). Sharp thresholds for high-dimensional and noisy sparsity recovery using l1-constrained quadratic programming (lasso)., IEEE Trans. Inf. Theory 55(5), 2183-2202. · Zbl 1367.62220
[36] Watson, G. N. (1933). Notes on generating functions of polynomials: (2) hermite polynomials., J. Lond. Math. Soc. s1-8(3), 194-199.
[37] Zhang, C.-H. and S. S. Zhang (2014). Confidence intervals for low dimensional parameters in high dimensional linear models., J. R. Statist. Soc. Ser. B 76(1), 217-242. · Zbl 1411.62196
[38] Zhang, X. and G. Cheng (2017). Simultaneous inference for high-dimensional linear models., J. Amer. Statist. Assoc. 112(518), 757-768.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.