Testing one hypothesis multiple times: the multidimensional case. (English) Zbl 07499262

Summary: The identification of new rare signals in data, the detection of a sudden change in a trend, and the selection of competing models are among the most challenging problems in statistical practice. These challenges can be tackled using a test of hypothesis where a nuisance parameter is present only under the alternative, and a computationally efficient solution can be obtained by the “testing one hypothesis multiple times” (TOHM) method. In the one-dimensional setting, a fine discretization of the space of the non identifiable parameter is specified, and a global \(p\)-value is obtained by approximating the distribution of the supremum of the resulting stochastic process. In this article, we propose a computationally efficient inferential tool to perform TOHM in the multidimensional setting. Here, the approximations of interest typically involve the expected Euler characteristics (EC) of the excursion set of the underlying random field. We introduce a simple algorithm to compute the EC in multiple dimensions and for arbitrarily large significance levels. This leads to an highly generalizable computational tool to perform hypothesis testing under nonstandard regularity conditions. Supplementary materials for this article are available online.


62-XX Statistics
Full Text: DOI arXiv


[1] Ackermann, M.; Ajello, M.; Albert, A.; Anderson, B.; Atwood, W. B.; Baldini, L.; Barbiellini, G.; Bastieri, D.; Bellazzini, R.; Bissaldi, E.; Blandford, R. D., “Updated Search for Spectral Lines From Galactic Dark Matter Interactions With Pass 8 Data From the Fermi Large Area Telescope,”, Physical Review D, 91, 122002 (2015)
[2] Adler, R., The Geometry of Random Fields (1981), Chichester: Wiley, Chichester
[3] Adler, R., “On Excursion Sets, Tube Formulas and Maxima of Random Fields,”, The Annals of Applied Probability, 10, 1-74 (2000) · Zbl 1171.60338
[4] Adler, R. J., Bartz, K., Kou, S. C., and Monod, A. (2017), “Estimating Thresholding Levels for Random Fields via Euler Characteristics,” arXiv no. 1704.08562.
[5] Adler, R.; Taylor, J., Random Fields and Geometry (2007), New York: Springer, New York · Zbl 1149.60003
[6] Algeri, S. (2019a), “Detecting New Signals Under Background Mismodelling,” arXiv no. 1906.06615.
[7] Algeri, S., “TOHM: Testing One Hypothesis Multiple Times,”, R Package Version 1.3 (2019)
[8] Algeri, S.; Conrad, J.; van Dyk, D., “A Method for Comparing Non-Nested Models With Application to Astrophysical Searches for New Physics,”, Monthly Notices of the Royal Astronomical Society: Letters, 458, L84-L88 (2016)
[9] Algeri, S.; van Dyk, D., “Testing One Hypothesis Multiple Times,”, Statistica Sinica, SS-, 2018-0027 (2019)
[10] Algeri, S.; van Dyk, D.; Conrad, J.; Anderson, B., “On Methods for Correcting for the Look-Elsewhere Effect in Searches for New Physics,”, Journal of Instrumentation, 11 (2016)
[11] Anderson, B.; Zimmer, S.; Conrad, J.; Gustafsson, M.; Sánchez-Conde, M.; Caputo, R., “Search for Gamma-Ray Lines Towards Galaxy Clusters With the Fermi-LAT,”, Journal of Cosmology and Astroparticle Physics, 2016 (2016)
[12] Benjamini, Y.; Hochberg, Y., “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing,”, Journal of the Royal Statistical Society, Series B, 57, 289-300 (1995) · Zbl 0809.62014
[13] Bonferroni, C. E., Studi in Onore del Professore Salvatore Ortu Carboni, “Il calcolo delle assicurazioni su gruppi di teste,”, 13-60 (1935), Rome: Bardi, Rome
[14] Bonferroni, C. E., “Teoria statistica delle classi e calcolo delle probabilità,”, Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze, 8, 3-62 (1936) · Zbl 0016.41103
[15] Bron, C.; Kerbosch, J., “Algorithm 457: Finding All Cliques of an Undirected Graph,”, Communications of the ACM, 16, 575-577 (1973) · Zbl 0261.68018
[16] Chen, Y.-C.; Genovese, C. R.; Wasserman, L., “A Comprehensive Approach to Mode Clustering,”, Electronic Journal of Statistics, 10, 210-241 (2016) · Zbl 1332.62200
[17] Chernoff, H., “On the Distribution of the Likelihood Ratio,”, The Annals of Mathematical Statistics, 25, 573-578 (1954) · Zbl 0056.37102
[18] Conrad, J., “Statistical Issues in Astrophysical Searches for Particle Dark Matter,”, Astroparticle Physics, 62, 165-177 (2015)
[19] Cramer, H., Mathematical Methods of Statistics (1946), Princeton, NJ: Princeton University Press, Princeton, NJ · Zbl 0063.01014
[20] Csardi, G.; Nepusz, T., “The igraph Software Package for Complex Network Research,”, Complex Systems, 1695 (2006)
[21] Davies, R. B., “Hypothesis Testing When a Nuisance Parameter Is Present Only Under the Alternative,”, Biometrika, 64, 247-254 (1977) · Zbl 0362.62026
[22] Davies, R. B., “Hypothesis Testing When a Nuisance Parameter Is Present Only Under the Alternatives,”, Biometrika, 74, 33-43 (1987) · Zbl 0612.62023
[23] Efron, B.; Tibshirani, R. J., An Introduction to the Bootstrap (1993), Boca Raton, FL: CRC Press, Boca Raton, FL · Zbl 0835.62038
[24] Eppstein, D.; Löffler, M.; Strash, D., Listing All Maximal Cliques in Sparse Graphs in Near-Optimal Time, 403-414 (2010), Berlin: Berlin, Heidelberg: Springer Berlin Heidelberg, Berlin: Berlin, Heidelberg · Zbl 1311.05187
[25] Ghosh, J.; Sen, P.; LeCam, L.; Olshen, R. A., Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, 2, On the Asymptotic Performance of the Log Likelihood Ratio Statistic for the Mixture Model and Related Results, 789-806 (1985), Monterey, CA: Wadsworth, Monterey, CA · Zbl 1373.62075
[26] Gonzalez-Estrada, E.; Villesanor-Alva, J. A. (2017)
[27] Gross, E.; Vitells, O., “Trial Factors for the Look Elsewhere Effect in High Energy Physics,”, The European Physical Journal C, 70, 525-530 (2010)
[28] Gruber, P. M., Convex and Discrete Geometry, 336 (2007), Berlin, Heidelberg: Springer Science & Business Media, Berlin, Heidelberg · Zbl 1139.52001
[29] Hasofer, A. M., “Upcrossings of Random Fields,”, Advances in Applied Probability, 10, 14-21 (1978) · Zbl 0387.60057
[30] Hochberg, Y., “A Sharper Bonferroni Procedure for Multiple Tests of Significance,”, Biometrika, 75, 800 (1988) · Zbl 0661.62067
[31] Johnston, H. C., “Cliques of a Graph-Variations on the Bron-Kerbosch Algorithm,”, International Journal of Computer & Information Sciences, 5, 209-238 (1976) · Zbl 0401.68042
[32] Lin, Y.; Lindsay, B. G., “Projections on Cones, Chi-Bar Squared Distributions, and Weyl’s Formula,”, Statistics & Probability Letters, 32, 367-376 (1997) · Zbl 0894.62058
[33] Lyons, L. (2013), “Discovering the Significance of 5 Sigma,” arXiv no. 1310.1284.
[34] Moran, P. A. P., “On Asymptotically Optimal Tests of Composite Hypotheses,”, Biometrika, 57, 47-55 (1970) · Zbl 0193.16708
[35] Muggeo, V. M., Segmented: An R Package to Fit Regression Models With Broken-Line Relationships,”, R News, 8, 20-25 (2008)
[36] Mukhopadhyay, S., “Large-Scale Mode Identification and Data-Driven Sciences,”, Electronic Journal of Statistics, 11, 215-240 (2017) · Zbl 1356.62052
[37] Oort, J., The Force Exerted by the Stellar System in the Direction Perpendicular to the Galactic Plane and Some Related Problems,”, Bulletin of the Astronomical Institutes of the Netherlands, 6, 249 (1932) · Zbl 0005.12805
[38] Pilla, R., and Loader, C. (2005), “Inference in Perturbation Models, Finite Mixtures and Scan Statistics: The Volume-of-Tube Formula,” arXiv no. math/0511503.
[39] Pilla, R. S.; Loader, C.; Taylor, C. C., “New Technique for Finding Needles in Haystacks: Geometric Approach to Distinguishing Between a New Source and Random Fluctuations,”, Physical Review Letters, 95, 230202 (2005)
[40] R Core Team, R: A Language and Environment for Statistical Computing (2019), Vienna, Austria: R Foundation for Statistical Computing, Vienna, Austria
[41] Razzano, M., High-Energy Gamma-Ray Astrophysics: From Solar Activity to Black Holes, “General Introduction to Fermi Tools (2013)
[42] Self, S. G.; Liang, K.-Y., “Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests Under Nonstandard Conditions,”, Journal of the American Statistical Association, 82, 605-610 (1987) · Zbl 0639.62020
[43] Shapiro, A., “Asymptotic Distribution of Test Statistics in the Analysis of Moment Structures Under Inequality Constraints,”, Biometrika, 72, 133-144 (1985) · Zbl 0596.62019
[44] Takemura, A.; Kuriki, S., “Weights of \(####\) Distribution for Smooth or Piecewise Smooth Cone Alternatives,”, The Annals of Statistics, 25, 2368-2387 (1997) · Zbl 0897.62055
[45] Taylor, J. E.; Adler, R. J., “Euler Characteristics for Gaussian Fields on Manifolds,”, The Annals of Probability, 31, 533-563 (2003) · Zbl 1026.60039
[46] Taylor, J.; Takemura, A.; Adler, R., “Validity of the Expected Euler Characteristic Heuristic,”, The Annals of Probability, 33, 1362-1396 (2005) · Zbl 1083.60031
[47] Taylor, J. E.; Worsley, K. J., “Random Fields of Multivariate Test Statistics, With Applications to Shape Analysis,”, The Annals of Statistics, 36, 1-27 (2008) · Zbl 1144.62083
[48] Taylor, J. E.; Worsley, K. J., “Detecting Sparse Cone Alternatives for Gaussian Random Fields, With an Application to FMRI,”, Statistica Sinica, 23, 1629-1656 (2013) · Zbl 1417.62119
[49] Vitells, O.; Gross, E., “Estimating the Significance of a Signal in a Multi-Dimensional Search,”, Astroparticle Physics, 35, 230-234 (2011)
[50] Weniger, C., “A Tentative Gamma-Ray Line From Dark Matter Annihilation at the Fermi Large Area Telescope,”, Journal of Cosmology and Astroparticle Physics, 2012, 007 (2012)
[51] Wilks, S. S., “The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses,”, Annals of Mathematical Statistics, 9, 60-62 (1938) · Zbl 0018.32003
[52] Worsley, K., “Local Maxima and the Expected Euler Characteristic of Excursion Sets of \(####\), F and t Fields,”, Advances in Applied Probability, 26, 13-42 (1994) · Zbl 0797.60042
[53] Worsley, K., “Estimating the Number of Peaks in a Random Field Using the Hadwiger Characteristic of Excursion Sets, With Applications to Medical Images,”, The Annals of Statistics, 23, 640-669 (1995) · Zbl 0898.62120
[54] Zwicky, F., Spectral Displacement of Extra Galactic Nebulae,”, Helvetica Physica Acta, 6, 110 (1933) · Zbl 0006.26906
[55] Zwicky, F., “On the Masses of Nebulae and of Clusters of Nebulae,”, The Astrophysical Journal, 86, 217 (1937) · Zbl 0017.28802
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.