A stochastic process approach to false discovery control. (English) Zbl 1092.62065

Genovese, Christopher; Wasserman, Larry

Ann. Stat. 32, No. 3, 1035-1061 (2004).

Summary: This paper extends the theory of false discovery rates (FDR) pioneered by Y. Benjamini and Y. Hochberg [J. R. Stat. Soc., Ser. B 57, No. 1, 289–300 (1995; Zbl 0809.62014)]. We develop a framework in which the False Discovery Proportion (FDP) – the number of false rejections divided by the number of rejections – is treated as a stochastic process. After obtaining the limiting distribution of the process, we demonstrate the validity of a class of procedures for controlling the False Discovery Rate (the expected FDP). We construct a confidence envelope for the whole FDP process. From these envelopes we derive confidence thresholds, for controlling the quantiles of the distribution of the FDP as well as controlling the number of false discoveries. We also investigate methods for estimating the \(p\)-value distribution.

MSC:

62H15	Hypothesis testing in multivariate analysis
62M99	Inference from stochastic processes
62E20	Asymptotic distribution theory in statistics
62G10	Nonparametric hypothesis testing

Keywords:

multiple testing; p-values; false discovery rate

Citations:

Zbl 0809.62014

References:

[1]	Abramovich, F., Benjamini, Y., Donoho, D. and Johnstone, I. (2000). Adapting to unknown sparsity by controlling the false discovery rate. Technical Report 2000-19, Dept. Statistics, Stanford Univ. · Zbl 1092.62005
[2]	Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289–300. · Zbl 0809.62014
[3]	Benjamini, Y. and Hochberg, Y. (2000). On the adaptive control of the false discovery rate in multiple testing with independent statistics. J. Educational and Behavioral Statistics 25 60–83.
[4]	Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165–1188. · Zbl 1041.62061 · doi:10.1214/aos/1013699998
[5]	Efron, B., Tibshirani, R., Storey, J. and Tusher, V. (2001). Empirical Bayes analysis of a microarray experiment. J. Amer. Statist. Assoc. 96 1151–1160. · Zbl 1073.62511 · doi:10.1198/016214501753382129
[6]	Finner, H. and Roters, M. (2002). Multiple hypotheses testing and expected number of type I errors. Ann. Statist. 30 220–238. · Zbl 1012.62020 · doi:10.1214/aos/1015362191
[7]	Genovese, C. R. and Wasserman, L. (2002). Operating characteristics and extensions of the FDR procedure. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 499–518. · Zbl 1090.62072 · doi:10.1111/1467-9868.00347
[8]	Havránek, T. and Chytil, M. (1983). Mechanizing hypotheses formation—a way for computerized exploratory data analysis? Bull. Internat. Statist. Inst. 50 104–121. · Zbl 0579.62003
[9]	Halperin, M., Lan, K. K. G. and Hamdy, M. I. (1988). Some implications of an alternative definition of the multiple comparison problem. Biometrika 75 773–778. · Zbl 0659.62077 · doi:10.1093/biomet/75.4.773
[10]	Hengartner, N. W. and Stark, P. B. (1995). Finite-sample confidence envelopes for shape-restricted densities. Ann. Statist. 23 525–550. JSTOR: · Zbl 0828.62043 · doi:10.1214/aos/1176324534
[11]	Hochberg, Y. and Benjamini, Y. (1990). More powerful procedures for multiple significance testing. Statistics in Medicine 9 811–818.
[12]	Hommel, G. and Hoffman, T. (1987). Controlled uncertainty. In Multiple Hypothesis Testing (P. Bauer, G. Hommel and E. Sonnemann, eds.) 154–161. Springer, Heidelberg.
[13]	Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures. Ann. Statist. 30 239–257. · Zbl 1101.62349 · doi:10.1214/aos/1015362192
[14]	Schweder, T. and Spj\otvoll, E. (1982). Plots of \(p\)-values to evaluate many tests simultaneously. Biometrika 69 493–502.
[15]	Storey, J. (2002). A direct approach to false discovery rates. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 479–498. · Zbl 1090.62073 · doi:10.1111/1467-9868.00346
[16]	Storey, J. (2003). The positive false discovery rate: A Bayesian interpretation and the \(q\)-value. Ann. Statist. 31 2013–2035. · Zbl 1042.62026 · doi:10.1214/aos/1074290335
[17]	Storey, J. D., Taylor, J. E. and Siegmund, D. (2004). Strong control, conservative point estimation, and simultaneous conservative consistency of false discovery rates: A unified approach. J. R. Stat. Soc. Ser. B Stat. Methodol. 66 187–205. · Zbl 1061.62110 · doi:10.1111/j.1467-9868.2004.00439.x
[18]	Swanepoel, J. W. H. (1999). The limiting behavior of a modified maximal symmetric \(2s\)-spacing with applications. Ann. Statist. 27 24–35. · Zbl 0937.62051 · doi:10.1214/aos/1018031099
[19]	van der Vaart, A. (1998). Asymptotic Statistics . Cambridge Univ. Press. · Zbl 0910.62001 · doi:10.1017/CBO9780511802256