Generalizing Simes’ test and Hochberg’s stepup procedure. (English) Zbl 1247.62193

Summary: In a multiple testing problem where one is willing to tolerate a few false rejections, a procedure controlling the familywise error rate (FWER) can potentially be improved in terms of its ability to detect false null hypotheses by generalizing it to control the \(k\)-FWER, the probability of falsely rejecting at least \(k\) null hypotheses, for some fixed \(k>1\). Simes’ test for testing the intersection null hypothesis is generalized to control the \(k\)-FWER weakly, that is, under the intersection null hypothesis, and Hochberg’s step up procedure for simultaneous testing of the individual null hypotheses is generalized to control the \(k\)-FWER strongly, that is, under any configuration of the true and false null hypotheses. The proposed generalizations are developed utilizing joint null distributions of the \(k\)-dimensional subsets of the \(p\)-values, assumed to be identical. The generalized Simes’ test is proved to control the \(k\)-FWER weakly under the multivariate totally positive of order two (MTP\(_{2})\) condition [S. Karlin and Y. Rinott, J. Multivariate Anal. 10, 467–498 (1980; Zbl 0469.60006)] of the joint null distribution of the \(p\)-values by generalizing the original Simes’ inequality. It is more powerful to detect \(k\) or more false null hypotheses than the original Simes’ test when the \(p\)-values are independent. A step down procedure strongly controlling the \(k\)-FWER, a version of the generalized Holm’s procedure, that is different from and more powerful than that of E.L. Lehmann and J.P. Romano [ibid. 33, No. 3, 1138–1154 (2005; Zbl 1072.62060)] with independent \(p\)-values, is derived before proposing the generalized Hochberg procedure. The strong control of the \(k\)-FWER for the generalized Hochberg procedure is established in situations where the generalized Simes test is known to control its \(k\)-FWER weakly.


62J15 Paired and multiple comparisons; multiple testing
Full Text: DOI arXiv Euclid


[1] Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165-1188. · Zbl 1041.62061
[2] Dudoit, S., Shaffer, J. P. and Boldrick, J. C. (2003). Multiple hypothesis testing in microarray experiments. Statist. Sci. 18 71-103. · Zbl 1048.62099
[3] Dudoit, S., van der Laan, M. and Pollard, K. (2002). Multiple testing: Part I. Single-step procedures for control of general type I error rates. Statist. Appl. Gen. Mol. Biol. 3 Article 13. · Zbl 1166.62338
[4] Guo, W. and Rao, M. B. (2006). On generalized closure principle for generalized familywise error rates. Unpublished report.
[5] Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75 800-802. JSTOR: · Zbl 0661.62067
[6] Hochberg, Y. and Tamhane, A. C. (1987). Multiple Comparison Procedures . Wiley, New York. · Zbl 0731.62125
[7] Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scand. J. Statist. 6 65-70. · Zbl 0402.62058
[8] Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75 383-386. · Zbl 0639.62025
[9] Hommel, G. and Hoffman, T. (1987). Controlled uncertainty. In Multiple Hypothesis Testing (P. Bauer, G. Hommel and E. Sonnemann, eds.) 154-161. Springer, Heidelberg.
[10] Karlin, S. and Rinott, Y. (1980). Classes of orderings of measures and related correlation inequalities I: Multivariate totally positive distributions. J. Multivariate Analysis 10 467-498. · Zbl 0469.60006
[11] Korn, E., Troendle, T., McShane, L. and Simon, R. (2004). Controlling the number of false discoveries: Application to high-dimensional genomic data. J. Statist. Plann. Inf. 124 279-398. · Zbl 1074.62070
[12] Lehmann, E. L. and Romano, J. P. (2005). Generalizations of the familywise error rate. Ann. Statist. 33 1138-1154. · Zbl 1072.62060
[13] Romano, J. P. and Shaikh, A. M. (2006). Stepup procedures for control of generalizations of the familywise error rate. Ann. Statist. 34 1850-1873. · Zbl 1246.62172
[14] Romano, J. P. and Wolf, M. (2005). Stepwise multiple testing as formalized data snooping. Econometrica 73 1237-1282. JSTOR: · Zbl 1153.62310
[15] Sarkar, S. K. (1998). Some probability inequalities for ordered MTP 2 random variables: A proof of the Simes conjecture. Ann. Statist. 26 494-504. · Zbl 0929.62065
[16] Sarkar, S. K. (2002). Some results on false discovery rate in stepwise multiple testing procedures. Ann. Statist. 30 239-257. · Zbl 1101.62349
[17] Sarkar, S. K. (2007). Stepup procedures controlling generalized FWER and generalized FDR. Ann. Statist. · Zbl 1129.62066
[18] Sarkar, S. K. and Chang, C.-K. (1997). Simes method for multiple hypothesis testing with positively dependent test rtatistics. J. Amer. Statist. Assoc. 92 1601-1608. JSTOR: · Zbl 0912.62079
[19] Simes, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73 751-754. JSTOR: · Zbl 0613.62067
[20] Tong, Y. L. (1989). Inequalities for a class of positively dependent random variables with a common marginal. Ann. Statist. 17 429-435. · Zbl 0679.60025
[21] van der Laan, M., Dudoit, S. and Pollard, K. (2004). Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives. Stat. Appl. Gen. Mol. Biol. 3 Article 15. · Zbl 1166.62379
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.