×
Coscrato, Victor; Izbicki, Rafael; Stern, Rafael B.

Agnostic tests can control the type I and type II errors simultaneously. (English) Zbl 1445.62009

Braz. J. Probab. Stat. 34, No. 2, 230-250 (2020).
Summary: Despite its common practice, statistical hypothesis testing presents challenges in interpretation. For instance, in the standard frequentist framework there is no control of the type II error. As a result, the non-rejection of the null hypothesis \((H_0)\) cannot reasonably be interpreted as its acceptance. We propose that this dilemma can be overcome by using agnostic hypothesis tests, since they can control the type I and II errors simultaneously. In order to make this idea operational, we show how to obtain agnostic hypothesis in typical models. For instance, we show how to build (unbiased) uniformly most powerful agnostic tests and how to obtain agnostic tests from standard \(p\)-values. Also, we present conditions such that the above tests can be made logically coherent. Finally, we present examples of consistent agnostic hypothesis tests.

MSC:

62A01 Foundations and philosophical topics in statistics
62F03 Parametric hypothesis testing
62C25 Compound decision problems in statistical decision theory

Keywords:

hypothesis test; uniformly most powerful tests; logical consistency; three-decision problem

Software:

BayesFactor
PDF BibTeX XML Cite
\textit{V. Coscrato} et al., Braz. J. Probab. Stat. 34, No. 2, 230--250 (2020; Zbl 1445.62009)
Full Text: DOI arXiv Euclid
  • logo of hbz
  • logo of WorldCat

References:

[1] Berg, N. (2004). No-decision classification: An alternative to testing for statistical significance. The Journal of Socio-Economics 33, 631-650.
[2] Bickel, P. J. and Doksum, K. A. (2015). Mathematical Statistics: Basic Ideas and Selected Topics, Volume I, Vol. 117. CRC Press. · Zbl 1380.62002
[3] Casella, G. and Berger, R. L. (2002). Statistical Inference, Vol. 2. Pacific Grove, CA: Duxbury. · Zbl 0699.62001
[4] Cohen, J. (1977). Chapter 9—F tests of variance proportions in multiple regression/correlation analysis. In Statistical Power Analysis for the Behavioral Sciences, Revised ed. (J. Cohen, ed.) 407-453. Academic Press.
[5] DeGroot, M. H. and Schervish, M. J. (2002). Probability and Statistics. Addison-Wesley.
[6] Esteves, L. G., Izbicki, R., Stern, J. M. and Stern, R. B. (2016). The logical consistency of simultaneous agnostic hypothesis tests. Entropy 18, 256.
[7] Geisser, S. and Johnson, W. O. (2006). Modes of Parametric Statistical Inference, Vol. 529. John Wiley & Sons.
[8] Izbicki, R. and Esteves, L. G. (2015). Logical consistency in simultaneous statistical test procedures. Logic Journal of the IGPL 23, 732-758. · Zbl 1405.62105
[9] Jeske, D. R., Linehan, J. A., Wilson, T. G., Kawachi, M. H., Wittig, K., Lamparska, K., Amparo, C., Mejia, R., Lai, F., Georganopoulou, D. and Steven, S. S. (2017). Two-stage classifiers that minimize pca3 and the psa proteolytic activity testing in the prediction of prostate cancer recurrence after radical prostatectomy. The Canadian Journal of Urology 24, 9089-9097.
[10] Jeske, D. R. and Smith, S. (2017). Maximizing the usefulness of statistical classifiers for two populations with illustrative applications. In Statistical Methods in Medical Research.
[11] Kass, R. E. and Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association 90, 773-795. · Zbl 0846.62028
[12] Lehmann, E. L. and Romano, J. P. (2006). Testing Statistical Hypotheses. Springer. · Zbl 1076.62018
[13] Lei, J. (2014). Classification with confidence. Biometrika 101, 755-769. · Zbl 1306.62143
[14] Levine, T. R., Weber, R., Park, H. S. and Hullett, C. R. (2008). A communication researchers’ guide to null hypothesis significance testing and alternatives. Human Communication Research 34, 188-209.
[15] Monahan, J. F. (2008). A Primer on Linear Models. CRC Press. · Zbl 1152.62043
[16] Morey, R. D. and Rouder, J. N. (2018). BayesFactor: Computation of Bayes factors for common designs. R package version 0.9.12-4.2. https://CRAN.R-project.org/package=BayesFactor.
[17] Mosteller, F. and Tukey, J. W. (1977). Data Analysis and Regression: A Second Course in Statistics. Addison-Wesley Series in Behavioral Science: Quantitative Methods.
[18] Neter, J., Kutner, M. H., Nachtsheim, C. J. and Wasserman, W. (1996). Applied Linear Statistical Models, Vol. 4. Chicago: Irwin.
[19] Neyman, J. (1976). Tests of statistical hypotheses and their use in studies of natural phenomena. Communications in Statistics Theory and Methods 5, 737-751. · Zbl 0333.62009
[20] Sadinle, M., Lei, J. and Wasserman, L. (2019). Least ambiguous set-valued classifiers with bounded error levels. Journal of the American Statistical Association. 114, 223-234. · Zbl 1478.62175
[21] Shao, J. (2003). Mathematical Statistics, Vol. 2. Springer. · Zbl 1018.62001
[22] Stern, J. M. (2011). Symmetry, invariance and ontology in physics and statistics. Symmetry 3, 611-635. · Zbl 1360.00084
[23] Stern, J. M. (2017). Continuous versions of haack’s puzzles: Equilibria, eigen-states and ontologies. Logic Journal of the IGPL 25, 604-631. · Zbl 1405.03021
[24] Stern, J.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.