False discovery and false nondiscovery rates in single-step multiple testing procedures. (English) Zbl 1091.62060

Sarkar, Sanat K.

Ann. Stat. 34, No. 1, 394-415 (2006).

Summary: Results on the false discovery rate (FDR) and the false nondiscovery rate (FNR) are developed for single-step multiple testing procedures. In addition to verifying desirable properties of FDR and FNR as measures of error rates, these results extend previously known results, providing further insights, particularly under dependence, into the notions of FDR and FNR and related measures.
First, considering fixed configurations of true and false null hypotheses, inequalities are obtained to explain how an FDR- or FNR-controlling single-step procedure, such as a Bonferroni or Šidák procedure, can potentially be improved. Two families of procedures are then constructed, one that modifies the FDR-controlling and the other that modifies the FNR-controlling Šidák procedure. These are proved to control FDR or FNR under independence less conservatively than the corresponding families that modify the FDR- or FNR-controlling Bonferroni procedure. Results of numerical investigations of the performance of the modified Šidák FDR procedure over its competitors are presented. Second, considering a mixture model where different configurations of true and false null hypotheses are assumed to have certain probabilities, results are also derived that extend some of J. D. Storey’s work [ibid. 31, No. 6, 2013–2035 (2003; Zbl 1042.62026); J. R. Stat. Soc., Ser. B 64, No. 3, 479–498 (2002; Zbl 1090.62073)] to the dependence case.

MSC:

62J15	Paired and multiple comparisons; multiple testing
62H99	Multivariate analysis
65C60	Computational problems in statistics (MSC2010)
62H15	Hypothesis testing in multivariate analysis

Keywords:

modified Bonferroni and Šidák procedures; mixture model; positive false discovery rate; positive false nondiscovery rate

Citations:

Zbl 1042.62026; Zbl 1090.62073

References:

[1]	Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289–300. · Zbl 0809.62014
[2]	Benjamini, Y. and Hochberg, Y. (2000). On the adaptive control of the false discovery rate in multiple testing with independent statistics. J. Educational and Behavioral Statistics 25 60–83.
[3]	Benjamini, Y., Krieger, A. M. and Yekutieli, D. (2002). Adaptive linear step-up false discovery rate controlling procedures. Unpublished manuscript. · Zbl 1108.62069
[4]	Benjamini, Y. and Liu, W. (1999). A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence. J. Statist. Plann. Inference 82 163–170. · Zbl 1063.62558 · doi:10.1016/S0378-3758(99)00040-3
[5]	Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependence. Ann. Statist. 29 1165–1188. · Zbl 1041.62061 · doi:10.1214/aos/1013699998
[6]	Efron, B. (2003). Robbins, empirical Bayes and microarrays. Ann. Statist. 31 366–378. · Zbl 1038.62099 · doi:10.1214/aos/1051027871
[7]	Efron, B., Tibshirani, R., Storey, J. D. and Tusher, V. (2001). Empirical Bayes analysis of a microarray experiment. J. Amer. Statist. Assoc. 96 1151–1160. · Zbl 1073.62511 · doi:10.1198/016214501753382129
[8]	Genovese, C. and Wasserman, L. (2002). Operating characteristics and extensions of the false discovery rate procedure. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 499–517. · Zbl 1090.62072 · doi:10.1111/1467-9868.00347
[9]	Genovese, C. and Wasserman, L. (2004). A stochastic process approach to false discovery control. Ann. Statist. 32 1035–1061. · Zbl 1092.62065 · doi:10.1214/009053604000000283
[10]	Hochberg, Y. and Benjamini, Y. (1990). More powerful procedures for multiple significance testing. Statistics in Medicine 9 811–818.
[11]	Karlin, S. (1968). Total Positivity 1 . Stanford Univ. Press. · Zbl 0219.47030
[12]	Lehmann, E. (1986). Testing Statistical Hypotheses , 2nd ed. Wiley, New York. · Zbl 0608.62020
[13]	Sarkar, S. K. (1998). Some probability inequalities for ordered MTP\(_2\) random variables: A proof of the Simes conjecture. Ann. Statist. 26 494–504. · Zbl 0929.62065 · doi:10.1214/aos/1028144846
[14]	Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures. Ann. Statist. 30 239–257. · Zbl 1101.62349 · doi:10.1214/aos/1015362192
[15]	Sarkar, S. K. (2004). FDR-controlling stepwise procedures and their false negatives rates. J. Statist. Plann. Inference 125 119–137. · Zbl 1097.62062 · doi:10.1016/j.jspi.2003.06.019
[16]	Simes, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73 751–754. · Zbl 0613.62067 · doi:10.1093/biomet/73.3.751
[17]	Storey, J. D. (2002). A direct approach to false discovery rates. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 479–498. · Zbl 1090.62073 · doi:10.1111/1467-9868.00346
[18]	Storey, J. D. (2003). The positive false discovery rate: A Bayesian interpretation and the \(q\)-value. Ann. Statist. 31 2013–2035. · Zbl 1042.62026 · doi:10.1214/aos/1074290335
[19]	Storey, J. D., Taylor, J. E. and Siegmund, D. (2004). Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach. J. R. Stat. Soc. Ser. B Stat. Methodol. 66 187–205. · Zbl 1061.62110 · doi:10.1111/j.1467-9868.2004.00439.x