Decision theory results for one-sided multiple comparison procedures. (English) Zbl 1066.62009

Summary: A resurgence of interest in multiple hypothesis testing has occurred in the last decade. Motivated by studies in genomics, microarrays, DNA sequencing, drug screening, clinical trials, bioassays, education and psychology, statisticians have been devoting considerable research energy in an effort to properly analyze multiple endpoint data. In response to new applications, new criteria and new methodology, many ad hoc procedures have emerged. The classical requirement has been to use procedures which control the strong familywise error rate (FWE) at some predetermined level \(\alpha\). That is, the probability of any false rejection of a true null hypothesis should be less than or equal to \(\alpha\). Finding desirable and powerful multiple test procedures is difficult under this requirement.
One of the more recent ideas is concerned with controlling the false discovery rate (FDR), that is, the expected proportion of rejected hypotheses which are, in fact, true. Many multiple test procedures do control the FDR. A much earlier approach to multiple testing was formulated by E. L. Lehmann [Ann. Math. Stat. 23, 541–552 (1952; Zbl 0048.11702); ibid. 28, 1–25 (1957; Zbl 0078.33402)]. Lehmann’s approach is decision theoretic and he treats the multiple endpoints problem as a \(2^k\) finite action problem when there are \(k\) endpoints. This approach is appealing since unlike the FWE and FDR criteria, the finite action approach pays attention to false acceptances as well as false rejections.
We view the multiple endpoints problem as a \(2^k\) finite action problem. We study the popular procedures single-step, step-down and step-up from the point of view of admissibility, Bayes and limit of Bayes properties. For our model, which is a prototypical one, and our loss function, we are able to demonstrate the following results under some fairly general conditions to be specified:
(i) The single-step procedure is admissible. (ii) A sequence of prior distributions is given for which the step down procedure is a limit of a sequence of Bayes procedures. (iii) For a vector risk function where each component is the risk for an individual testing problem, various admissibility and inadmissibility results are obtained.
In a companion paper [see the following entry, ibid. 33, 145–158 (2005; Zbl 1066.62009)], we are able to give a characterization of Bayes procedures and their limits. The characterization yields a complete class and the additional useful result that the step-up procedure is inadmissible. The inadmissibility of step-up is demonstrated there for a more stringent loss function. Additional decision theoretic type results are also obtained in this paper.


62C10 Bayesian problems; characterization of Bayes procedures
62C15 Admissibility in statistical decision theory
62J15 Paired and multiple comparisons; multiple testing
62C07 Complete class results in statistical decision theory
62F03 Parametric hypothesis testing
Full Text: DOI arXiv


[1] Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289–300. · Zbl 0809.62014
[2] Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165–1188. · Zbl 1041.62061 · doi:10.1214/aos/1013699998
[3] Cohen, A. and Sackrowitz, H. B. (1984). Decision theory results for vector risks with applications. Statist. Decisions Suppl. 1 159–176. · Zbl 0554.62006
[4] Cohen, A. and Sackrowitz, H. B. (2004). Monotonicity properties of multiple endpoint testing procedures. J. Statist. Plann. Inference 125 17–30. · Zbl 1074.62011 · doi:10.1016/j.jspi.2003.10.008
[5] Cohen, A. and Sackrowitz, H. B. (2005). Characterization of Bayes procedures for multiple endpoint problems and inadmissibility of the step-up procedure. Ann. Statist. 33 145–158. · Zbl 1066.62010 · doi:10.1214/009053604000000986
[6] Dudoit, S., Shaffer, J. P. and Boldrick, J. C. (2003). Multiple hypothesis testing in microarray experiments. Statist. Sci. 18 71–103. · Zbl 1048.62099 · doi:10.1214/ss/1056397487
[7] Efron, B., Tibshirani, R., Storey, J. D. and Tusher, V. (2001). Empirical Bayes analysis and a microarray experiment. Technical Report 216, Dept. Statistics, Stanford Univ. · Zbl 1073.62511 · doi:10.1198/016214501753382129
[8] Finner, H. and Roters, M. (2002). Multiple hypotheses testing and expected number of type I errors. Ann. Statist. 30 220–238. · Zbl 1012.62020 · doi:10.1214/aos/1015362191
[9] Finner, H. and Strassburger, K. (2002). The partitioning principle: A powerful tool in multiple decision theory. Ann. Statist. 30 1194–1213. · Zbl 1029.62064 · doi:10.1214/aos/1031689023
[10] Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75 800–802. · Zbl 0661.62067 · doi:10.1093/biomet/75.4.800
[11] Hochberg, Y. and Tamhane, A. C. (1987). Multiple Comparison Procedures . Wiley, New York. · Zbl 0731.62125
[12] Krishnaiah, P. R. and Pathak, P. K. (1967). Tests for the equality of covariance matrices under the intraclass correlation model. Ann. Math. Statist. 38 1286–1288. · Zbl 0166.15201 · doi:10.1214/aoms/1177698801
[13] Lehmann, E. L. (1952). Testing multiparameter hypotheses. Ann. Math. Statist. 23 541–552. · Zbl 0048.11702 · doi:10.1214/aoms/1177729333
[14] Lehmann, E. L. (1957). A theory of some multiple decision problems. I. Ann. Math. Statist. 28 1–25. · Zbl 0078.33402 · doi:10.1214/aoms/1177707034
[15] Lehmann, E. L. (1986). Testing Statistical Hypotheses , 2nd ed. Wiley, New York. · Zbl 0608.62020
[16] Marcus, R., Peritz, E. and Gabriel, K. R. (1976). On closed testing procedures with special reference to ordered analysis of variance. Biometrika 63 655–660. · Zbl 0353.62037 · doi:10.1093/biomet/63.3.655
[17] Marden, J. I. (1982). Minimal complete classes of tests of hypotheses with multivariate one-sided alternatives. Ann. Statist. 10 962–970. JSTOR: · Zbl 0545.62038 · doi:10.1214/aos/1176345886
[18] Matthes, T. K. and Truax, D. R. (1967). Tests of composite hypotheses for the multivariate exponential family. Ann. Math. Statist. 38 681–697. · Zbl 0152.17802 · doi:10.1214/aoms/1177698862
[19] Mood, A. M., Graybill, F. A. and Boes, D. C. (1974). Introduction to the Theory of Statistics , 3rd ed. McGraw-Hill, New York. · Zbl 0277.62002
[20] Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference . Wiley, New York. · Zbl 0645.62028
[21] Sarkar, S. (2002). Some results on false discovery rate in stepwise multiple testing procedures. Ann. Statist. 30 239–257. · Zbl 1101.62349 · doi:10.1214/aos/1015362192
[22] Shaffer, J. P. (1995). Multiple hypothesis testing. Annual Review of Psychology 46 561–584.
[23] Stefánsson, G., Kim, W. and Hsu, J. C. (1988). On confidence sets in multiple comparisons. In Statistical Decision Theory and Related Topics IV (S. S. Gupta and J. O. Berger, eds.) 2 89–104. Springer, New York. · Zbl 0685.62034
[24] Stein, C. M. (1956). The admissibility of Hotelling’s \(T^2\)-test. Ann. Math. Statist. 27 616–623. · Zbl 0073.14301 · doi:10.1214/aoms/1177728171
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.