Asymptotic properties of false discovery rate controlling procedures under independence. (English) Zbl 1320.62181

Neuvial, Pierre

Electron. J. Stat. 2, 1065-1110 (2008); corrigendum ibid. 3, 1083 (2009).

Summary: We investigate the performance of a family of multiple comparison procedures for strong control of the False Discovery Rate (FDR). The FDR is the expected False Discovery Proportion (FDP), that is, the expected fraction of false rejections among all rejected hypotheses. A number of refinements to the original Benjamini-Hochberg procedure [Y. Benjamini and Y. Hochberg, J. R. Stat. Soc., Ser. B 57, No. 1, 289–300 (1995; Zbl 0809.62014)] have been proposed, to increase power by estimating the proportion of true null hypotheses, either implicitly, leading to one-stage adaptive procedures [G. Blanchard and É. Roquain, J. Mach. Learn. Res. 10, 2837–2871 (2009; Zbl 1235.62093); H. Finner et al., Ann. Stat. 37, No. 2, 596–618 (2009; Zbl 1162.62068)] or explicitly, leading to two-stage adaptive (or plug-in) procedures [Y. Benjamini et al., Biometrika 93, No. 3, 491–507 (2006; Zbl 1108.62069)]; J. D. Storey, J. R. Stat. Soc., Ser. B, Stat. Methodol. 64, No. 3, 479–498 (2002; Zbl 1090.62073)].
We use a variant of the stochastic process approach proposed by C. Genovese and L. Wasserman [Ann. Stat. 32, No. 3, 1035–1061 (2004; Zbl 1092.62065)] to study the fluctuations of the FDP achieved with each of these procedures around its expectation, for independent tested hypotheses.
We introduce a framework for the derivation of generic Central Limit Theorems for the FDP of these procedures, characterizing the associated regularity conditions, and comparing the asymptotic power of the various procedures. We interpret recently proposed one-stage adaptive procedures [Blanchard and Roquain, loc. cit.; Finner et al., loc. cit.] as fixed points in the iteration of well known two-stage adaptive procedures [Benjamini et al., loc. cit.; Storey, loc. cit.].

MSC:

62J15	Paired and multiple comparisons; multiple testing
62H15	Hypothesis testing in multivariate analysis
62G10	Nonparametric hypothesis testing
62G20	Asymptotic properties of nonparametric inference
60F05	Central limit and other weak theorems

Keywords:

multiple hypothesis testing; Benjamini-Hochberg procedure; FDP; FDR

Citations:

Zbl 0809.62014; Zbl 1235.62093; Zbl 1162.62068; Zbl 1108.62069; Zbl 1090.62073; Zbl 1092.62065

Software:

SAM

References:

[1]	Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing., J. R. Stat. Soc. Ser. B Stat. Methodol. 57 , 1, 289-300. · Zbl 0809.62014
[2]	Benjamini, Y., Krieger, A. M., and Yekutieli, D. (2006). Adaptive linear step-up procedures that control the false discovery rate., Biometrika 93 , 3, 491-507. · Zbl 1108.62069 · doi:10.1093/biomet/93.3.491
[3]	Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency., Ann. Statist. 29 , 4, 1165-1188. · Zbl 1041.62061 · doi:10.1214/aos/1013699998
[4]	Blanchard, G. and Roquain, E. (2008). Adaptive FDR control under independence and dependence. arXiv preprint, math.ST/0707.0536v2. · Zbl 1320.62179 · doi:10.1214/08-EJS180
[5]	Chi, Z. (2007). On the performance of FDR control: constraints and a partial solution., Ann. Statist. 35 , 4, 1409-1431. · Zbl 1125.62075 · doi:10.1214/009053607000000037
[6]	Donsker, M. D. (1951). An invariance principle for certain probability limit theorems., Mem. Amer. Math. Soc. 6 , 12. · Zbl 0042.37602
[7]	Finner, H., Dickhaus, T., and Roters, M. (2008). On the false discovery rate and an asymptotically optimal rejection curve., Ann. Statist. (to appear). · Zbl 1162.62068 · doi:10.1214/07-AOS569
[8]	Finner, H. and Roters, M. (2001). On the false discovery rate and expected type I errors., Biom. J. 43 , 8, 985-1005. · Zbl 0989.62061 · doi:10.1002/1521-4036(200112)43:8<985::AID-BIMJ985>3.0.CO;2-4
[9]	Finner, H. and Roters, M. (2002). Multiple hypotheses testing and expected number of type I errors., Ann. Statist. 30 , 1, 220-238. · Zbl 1012.62020 · doi:10.1214/aos/1015362191
[10]	Genovese, C. and Wasserman, L. (2002). Operating characteristics and extensions of the false discovery rate., J. R. Stat. Soc. Ser. B Stat. Methodol. 64 , 3, 499-517. · Zbl 1090.62072 · doi:10.1111/1467-9868.00347
[11]	Genovese, C. and Wasserman, L. (2004). A stochastic process approach to false discovery control., Ann. Statist. 32 , 3, 1035-1061. · Zbl 1092.62065 · doi:10.1214/009053604000000283
[12]	Genovese, C. and Wasserman, L. (2006). Exceedance control of the false discovery proportion., J. Amer. Statist. Assoc. 101 , 476, 1408-1417. · Zbl 1171.62338 · doi:10.1198/016214506000000339
[13]	Lehmann, E. L. and Romano, J. P. (2005). Generalizations of the familywise error rate., Ann. Statist 33 , 3, 1138-1154. · Zbl 1072.62060 · doi:10.1214/009053605000000084
[14]	Neuvial, P. (2008). Intrinsic bounds and false discovery rate control in multiple testing problems. In revision for, J. Mach. Learn. Res. · Zbl 1320.62181
[15]	Perone Pacifico, M., Genovese, C., Verdinelli, I., and Wasserman, L. (2004). False discovery control for random fields., J. Amer. Statist. Assoc. 99 , 468, 1002-1014. · Zbl 1055.62105 · doi:10.1198/0162145000001655
[16]	Romano, J. P. and Shaikh, A. M. (2006a). On stepdown control of the false discovery proportion. In, Optimality: The Second Erich L. Lehmann Symposium , J. Rojo, Ed. Vol. 33 . Institute of Mathematical Statistics, Beachwood, Ohio, USA. · Zbl 1268.62078 · doi:10.1214/074921706000000383
[17]	Romano, J. P. and Shaikh, A. M. (2006b). Step-up procedures for control of generalizations of the family-wise error rate., Ann. Statist. 34 , 4, 1850-1873. · Zbl 1246.62172 · doi:10.1214/009053606000000461
[18]	Romano, J. P. and Wolf, M. (2007). Control of generalized error rates in multiple testing., Ann. Statist. 35 , 4, 1378-1408. · Zbl 1127.62063 · doi:10.1214/009053606000001622
[19]	Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures., Ann. Statist. 30 , 1, 239-257. · Zbl 1101.62349 · doi:10.1214/aos/1015362192
[20]	Simes, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance., Biometrika 73 , 3, 751-754. · Zbl 0613.62067 · doi:10.1093/biomet/73.3.751
[21]	Storey, J. D. (2002). A direct approach to false discovery rates., J. R. Stat. Soc. Ser. B Stat. Methodol. 64 , 3, 479-498. · Zbl 1090.62073 · doi:10.1111/1467-9868.00346
[22]	Storey, J. D. (2003). The positive false discovery rate: a bayesian interpretation and the q-value., Ann. Statist. 31 , 6, 2013-2035. · Zbl 1042.62026 · doi:10.1214/aos/1074290335
[23]	Storey, J. D., Taylor, J. E., and Siegmund, D. (2004). Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach., J. R. Stat. Soc. Ser. B Stat. Methodol. 66 , 1, 187-205. · Zbl 1061.62110 · doi:10.1111/j.1467-9868.2004.00439.x
[24]	Storey, J. D. and Tibshirani, R. (2003). SAM thresholding and false discovery rates for detecting differential gene expression in DNA microarrays. In, The Analysis of Gene Expression Data: Methods and Software . Springer, New York, 272-290. · doi:10.1007/0-387-21679-0_12
[25]	Sun, W. and Cai, T. T. (2007). Oracle and adaptive compound decision rules for false discovery rate control., J. Amer. Statist. Assoc. 102 , 479, 901-912. · Zbl 1469.62318 · doi:10.1198/016214507000000545
[26]	van der Laan, M. J., Dudoit, S., and Pollard, K. S. (2004). Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives., Stat. Appl. Genet. Mol. Biol. 3 , Art. 15, 27 pp. · Zbl 1166.62379
[27]	van der Vaart, A. W. (1998)., Asymptotic Statistics . Cambridge University Press. · Zbl 0910.62001
[28]	Wu, W. B. (2008). On false discovery control under dependence., Ann. Statist. 36 , 1, 364-380. · Zbl 1139.62040 · doi:10.1214/009053607000000730