FDR control with adaptive procedures and FDR monotonicity. (English) Zbl 1232.62106

Zeisel, Amit; Zuk, Or; Domany, Eytan

Ann. Appl. Stat. 5, No. 2A, 943-968 (2011).

Summary: The steep rise in availability and usage of high-throughput technologies in biology brought with it a clear need for methods to control the False Discovery Rate (FDR) in multiple tests. Y. Benjamini and Y. Hochberg (BH) [J. R. Stat. Soc., Ser. B 57, No. 1, 289–300 (1995; Zbl 0809.62014)] introduced a simple procedure and proved that it provides a bound on the expected value, \(FDR \leq q\). Since then, many authors tried to improve the BH bound, with one approach being designed as an adaptive procedures, which aims at estimating the number of true null hypotheses in order to get a better FDR bound. Our two main rigorous results are the following:
(i) a theorem that provides a bound on the FDR for adaptive procedures that use any estimator for the number of true hypotheses \((m_{0})\), (ii) a theorem that proves a monotonicity property of general BH-like procedures, both for the case where the hypotheses are independent.
We also propose two improved procedures for which we prove the FDR control for the independent case, and demonstrate their advantages over several available bounds, on simulated data and on a large number of gene expression data sets. Both applications are simple and involve a similar amount of computation as the original BH procedure. We compare the performance of our proposed procedures with BH and other procedures and find that in most cases we get more power for the same level of statistical significance.

MSC:

62J15	Paired and multiple comparisons; multiple testing
65C60	Computational problems in statistics (MSC2010)
62P10	Applications of statistics to biology and medical sciences; meta analysis
92C50	Medical applications (general)

Keywords:

false discovery rate; improved BH; gene expression analysis

Citations:

Zbl 0809.62014

Software:

Oncomine

References:

[1]	Andersson, A., Ritz, C., Lindgren, D., Edén, P., Lassen, C., Heldrup, J., Olofsson, T., Råde, J., Fontes, M., Porwit-Macdonald, A., Behrendtz, M., Höglund, M., Johansson, B. and Fioretos, T. (2007). Microarray-based classification of a consecutive series of 121 childhood acute leukemias: Prediction of leukemic and genetic subtype as well as of minimal residual disease status. Leukemia 21 1198-1203.
[2]	Aven, T. and Jensen, U. (1999). Stochastic Models in Reliability . Springer, New York. · Zbl 0926.60006 · doi:10.1007/b97596
[3]	Basso, K., Margolin, A. A., Stolovitzky, G., Klein, U., Dalla-Favera, R. and Califano, A. (2005). Reverse engineering of regulatory networks in human B cells. Nat. Genet. 37 382-390.
[4]	Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300. · Zbl 0809.62014
[5]	Benjamini, Y., Krieger, A. M. and Yekutieli, D. (2006). Adaptive linear step-up procedures that control the false discovery rate. Biometrica 93 491-507. · Zbl 1108.62069 · doi:10.1093/biomet/93.3.491
[6]	Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165-1168. · Zbl 1041.62061 · doi:10.1214/aos/1013699998
[7]	Bittner, M. (2005). A window on the dynamics of biological switches. Nat. Biotechnol. 23 183-184.
[8]	Bullinger, L., Döhner, K., Bair, E., Fröhling, S., Schlenk, R. F., Tibshirani, R., Döhner, H. and Pollack, J. R. (2004). Use of gene-expression profiling to identify prognostic subclasses in adult acute myeloid leukemia. N. Engl. J. Med. 350 1605-1616.
[9]	Choi, Y. L., Tsukasaki, K., O’Neill, M. C., Yamada, Y., Onimaru, Y., Matsumoto, K., Ohashi, J., Yamashita, Y., Tsutsumi, S., Kaneda, R., Takada, S., Aburatani, H., Kamihira, S., Nakamura, T., Tomonaga, M. and Mano, H. (2007). A genomic analysis of adult T-cell leukemia. Oncogene 26 1245-1255.
[10]	Chowdary, D., Lathrop, J., Skelton, J., Curtin, K., Briggs, T., Zhang, Y., Yu, J., Wang, Y. and Mazumder, A. (2006). Prognostic gene expression signatures can be measured in tissues collected in RNA later preservative. J. Mol. Diagn. 8 31-39.
[11]	Gavrilov, Y., Benjamini, Y. and Sarkar, S. K. (2009). An adaptive step-down procedure with proven FDR control under independence. Ann. Statist. 37 619-629. · Zbl 1162.62069 · doi:10.1214/07-AOS586
[12]	Genovese, C. and Wasserman, L. (2002). Operating characteristics and extensions of the false discovery rate procedure. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 499-517. · Zbl 1090.62072 · doi:10.1111/1467-9868.00347
[13]	Graudens, E., Boulanger, V., Mollard, C., Mariage-Samson, R., Barlet, X., Grémy, G., Couillault, C., Lajémi, M., Piatier-Tonneau, D., Zaborski, P., Eveno, E., Auffray, C. and Imbeaud, S. (2006). Deciphering cellular states of innate tumor drug responses. Genome Biol. 7 R19-R19.
[14]	Koinuma, K., Yamashita, Y., Liu, W., Hatanaka, H., Kurashina, K., Wada, T., Takada, S., Kaneda, R., Choi, Y. L., Fujiwara, S.-I., Miyakura, Y., Nagai, H. and Mano, H. (2006). Epigenetic silencing of AXIN2 in colorectal carcinoma with microsatellite instability. Oncogene 25 139-146.
[15]	Laiho, P., Kokko, A., Vanharanta, S., Salovaara, R., Sammalkorpi, H., Järvinen, H., Mecklin, J.-P., Karttunen, T. J., Tuppurainen, K., Davalos, V., Schwartz, S., Arango, D., Mäkinen, M. J. and Aaltonen, L. A. (2007). Serrated carcinomas form a subclass of colorectal cancer with distinct molecular basis. Oncogene 26 312-320.
[16]	Miller, L. D., Smeds, J., George, J., Vega, V. B., Vergara, L., Ploner, A., Pawitan, Y., Hall, P., Klaar, S., Liu, E. T. and Bergh, J. (2005). An expression signature for p53 status in human breast cancer predicts mutation status, transcriptional effects, and patient survival. Proc. Natl. Acad. Sci. USA 102 13550-13555.
[17]	Owen, A. B. (2005). Variance of the number of false discoveries. J. Roy. Statist. Soc. Ser. B 67 411-426. · Zbl 1069.62102 · doi:10.1111/j.1467-9868.2005.00509.x
[18]	Pawitan, Y., Bjöhle, J., Amler, L., Borg, A.-L., Egyhazi, S., Hall, P., Han, X., Holmberg, L., Huang, F., Klaar, S., Liu, E. T., Miller, L., Nordgren, H., Ploner, A., Sandelin, K., Shaw, P. M., Smeds, J., Skoog, L., Wedrén, S. and Bergh, J. (2005). Gene expression profiling spares early breast cancer patients from adjuvant therapy: Derived and validated in two population-based cohorts. Breast Cancer Res. 7 R953-R964.
[19]	Pounds, S. and Cheng, C. (2006). Robust estimation of the false discovery rate. Bioinformatics 22 1979-1987.
[20]	Reiner, A. (2007). FDR control by the BH procedure for two-sided correlated tests with implications to gene expression data analysis. Biom. J. 49 107-126. · doi:10.1002/bimj.200510313
[21]	Rhodes, D. R., Kalyana-Sundaram, S., Mahavisno, V., Varambally, R., Yu, J., Briggs, B. B., Barrette, T. R., Anstet, M. J., Kincead-Beal, C., Kulkarni, P., Varambally, S., Ghosh, D. and Chinnaiyan, A. M. (2007). Oncomine 3.0: Genes, pathways, and networks in a collection of 18,000 cancer gene expression profiles. Neoplasia 9 166-180.
[22]	Ross, M. E., Zhou, X., Song, G., Shurtleff, S. A., Girtman, K., Williams, W. K., Liu, H.-C., Mahfouz, R., Raimondi, S. C., Lenny, N., Patel, A. and Downing, J. R. (2003). Classification of pediatric acute lymphoblastic leukemia by gene expression profiling. Blood 102 2951-2959.
[23]	Storey, J. D. (2002). A direct approach to false discovery rate. J. Roy. Statist. Soc. Ser. B 64 479-498. · Zbl 1090.62073 · doi:10.1111/1467-9868.00346
[24]	Storey, J. D., Taylor, J. E. and Siegmund, D. (2004). Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: A unified approach. J. Roy. Statist. Soc. Ser. B 66 187-205. · Zbl 1061.62110 · doi:10.1111/j.1467-9868.2004.00439.x
[25]	Valk, P. J. M., Verhaak, R. G. W., Beijen, M. A., Erpelinck, C. A. J., Barjesteh van Waalwijk van Doorn-Khosrovani, S., Boer, J. M., Beverloo, H. B., Moorhouse, M. J., van der Spek, P. J., Löwenberg, B. and Delwel, R. (2004). Prognostically useful gene-expression profiles in acute myeloid leukemia. N. Engl. J. Med. 350 1617-1628.
[26]	van de Vijver, M. J., He, Y. D., van’t Veer, L. J., Dai, H., Hart, A. A. M., Voskuil, D. W., Schreiber, G. J., Peterse, J. L., Roberts, C., Marton, M. J., Parrish, M., Atsma, D., Witteveen, A., Glas, A., Delahaye, L., van der Velde, T., Bartelink, H., Rodenhuis, S., Rutgers, E. T., Friend, S. H. and Bernards, R. (2002). A gene-expression signature as a predictor of survival in breast cancer. N. Engl. J. Med. 347 1999-2009.
[27]	Wang, Y., Klijn, J. G. M., Zhang, Y., Sieuwerts, A. M., Look, M. P., Yang, F., Talantov, D., Timmermans, M., Meijer-van Gelder, M. E., Yu, J., Jatkoe, T., Berns, E. M. J. J., Atkins, D. and Foekens, J. A. (2005). Gene-expression profiles to predict distant metastasis of lymph-node-negative primary breast cancer. Lancet 365 671-679.
[28]	Watanabe, T., Kobunai, T., Toda, E., Yamamoto, Y., Kanazawa, T., Kazama, Y., Tanaka, J., Tanaka, T., Konishi, T., Okayama, Y., Sugimoto, Y., Oka, T., Sasaki, S., Muto, T. and Nagawa, H. (2006). Distal colorectal cancers with microsatellite instability (MSI) display distinct gene expression profiles that are different from proximal MSI cancers. Cancer Res. 66 9804-9808.
[29]	Yekutieli, D. and Benjamini, Y. (1999). Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. J. Statist. Plann. Inference 82 171-196. · Zbl 1063.62563 · doi:10.1016/S0378-3758(99)00041-5
[30]	Yeoh, E.-J., Ross, M. E., Shurtleff, S. A., Williams, W. K., Patel, D., Mahfouz, R., Behm, F. G., Raimondi, S. C., Relling, M. V., Patel, A., Cheng, C., Campana, D., Wilkins, D., Zhou, X., Li, J., Liu, H., Pui, C.-H., Evans, W. E., Naeve, C., Wong, L. and Downing, J. R. (2002). Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling. Cancer Cell 1 133-143.
[31]	Zeisel, A., Zuk, O. and Domany, E. (2010). Supplement to “FDR control with adaptive procedures and FDR monotonicity.” DOI: . · Zbl 1232.62106
[32]	Zhao, H., Langerød, A., Ji, Y., Nowels, K. W., Nesland, J. M., Tibshirani, R., Bukholm, I. K., Kåresen, R., Botstein, D., Børresen-Dale, A.-L. and Jeffrey, S. S. (2004). Different gene expression patterns in invasive lobular and ductal carcinomas of the breast. Mol. Biol. Cell 15 2523-2536.
[33]	Zou, T.-T., Selaru, F. M., Xu, Y., Shustova, V., Yin, J., Mori, Y., Shibata, D., Sato, F., Wang, S., Olaru, A., Deacu, E., Liu, T. C., Abraham, J. M. and Meltzer, S. J. (2002). Application of cDNA microarrays to generate a molecular taxonomy capable of distinguishing between colon cancer and normal colon. Oncogene 21 4855-4862.