×

An algorithm to compute the power of Monte Carlo tests with guaranteed precision. (English) Zbl 1347.62011

Summary: This article presents an algorithm that generates a conservative confidence interval of a specified length and coverage probability for the power of a Monte Carlo test (such as a bootstrap or permutation test). It is the first method that achieves this aim for almost any Monte Carlo test. Previous research has focused on obtaining as accurate a result as possible for a fixed computational effort, without providing a guaranteed precision in the above sense. The algorithm we propose does not have a fixed effort and runs until a confidence interval with a user-specified length and coverage probability can be constructed. We show that the expected effort required by the algorithm is finite in most cases of practical interest, including situations where the distribution of the \(p\)-value is absolutely continuous or discrete with finite support. The algorithm is implemented in the R-package simctest, available on CRAN.

MSC:

62-04 Software, source code, etc. for problems pertaining to statistics
62L12 Sequential estimation
62L15 Optimal stopping in statistics
62F40 Bootstrap, jackknife and other resampling methods

Software:

bootlib; CRAN; R; simctest
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Boos, D. and Zhang, J. (2000). Monte Carlo evaluation of resampling-based hypothesis tests. J. Amer. Statist. Assoc. 95 486-492.
[2] Clopper, C. and Pearson, E. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26 404-413. · JFM 60.1175.02
[3] Davison, A. C. and Hinkley, D. V. (1997). Bootstrap Methods and Their Application. Cambridge Series in Statistical and Probabilistic Mathematics 1 . Cambridge Univ. Press, Cambridge. · Zbl 0886.62001
[4] Embrechts, P., Klüppelberg, C. and Mikosch, T. (1997). Modelling Extremal Events : For Insurance and Finance. Applications of Mathematics ( New York ) 33 . Springer, Berlin. · Zbl 0873.62116
[5] Gandy, A. (2009). Sequential implementation of Monte Carlo tests with uniformly bounded resampling risk. J. Amer. Statist. Assoc. 104 1504-1511. · Zbl 1205.65016 · doi:10.1198/jasa.2009.tm08368
[6] Gandy, A. and Rubin-Delanchy, P. (2013). Supplement to “An algorithm to compute the power of Monte Carlo tests with guaranteed precision.” . · Zbl 1347.62011
[7] Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13-30. · Zbl 0127.10602 · doi:10.2307/2282952
[8] Keilson, J. and Geber, H. (1971). Some results for discrete unimodality. J. Amer. Statist. Assoc. 66 386-389. · Zbl 0236.60017 · doi:10.2307/2283941
[9] Keilson, J. and Sumita, U. (1982). Uniform stochastic ordering and related inequalities. Canad. J. Statist. 10 181-198. · Zbl 0516.60063 · doi:10.2307/3556181
[10] Oden, N. L. (1991). Allocation of effort in Monte Carlo simulation for power of permutation tests. J. Amer. Statist. Assoc. 86 1074-1076.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.