zbMATH — the first resource for mathematics

On comparison of experts. (English) Zbl 1429.91244
Summary: A policy maker faces a sequence of unknown outcomes. At each stage two (self-proclaimed) experts provide probabilistic forecasts on the outcome in the next stage. A comparison test is a protocol for the policy maker to (eventually) decide which of the two experts is better informed. The protocol takes as input the sequence of pairs of forecasts and actual outcomes and (weakly) ranks the two experts.
We focus on anonymous and non-counterfactual comparison tests and propose two natural properties to which such a comparison test must adhere. We show that these determine the test in an essentially unique way. The resulting test is a function of the derivative of the induced pair of measures at the realized outcomes.
91B82 Statistical methods; economic indices and measures
62M20 Inference from stochastic processes and prediction
60G25 Prediction theory (aspects of stochastic processes)
Full Text: DOI
[1] Al-Najjar, N.; Sandroni, A.; Smorodinsky, R.; Weinstein, J., Testing theories with learnable and predictive representations, J. Econ. Theory, 145, 6, 2203-2217 (2010) · Zbl 1203.62220
[2] Al-Najjar, N.; Weinstein, J., Comparative testing of experts, Econometrica, 76, 3, 541-559 (2008) · Zbl 1152.91703
[3] Billingsley, P., Probability and Measure (1995), New York · Zbl 0822.60002
[4] Dawid, P., The well-calibrated bayesian, J. Am. Stat. Assoc., 77, 605-613 (1982) · Zbl 0495.62005
[5] Dekel, E.; Feinberg, Y., Non-bayesian testing of a stochastic prediction, Rev. Econ. Stud., 73, 893-936 (2006) · Zbl 1104.62103
[6] Echenique, F., Shmaya, E., 2008. You won’t harm me if you fool me. Mimeo.
[7] Edwards, A., Likelihood (1972), Cambridge University Press · Zbl 0231.62005
[8] Feinberg, Y.; Stewart, C., Testing multiple forecasters, Econometrica, 76, 561-582 (2008) · Zbl 1152.91708
[9] Fortnow, L.; Vohra, R., The complexity of forecast testing, Econometrica, 77, 93-105 (2009) · Zbl 1160.91396
[10] Foster, D.; Vohra, R., Asymptotic calibration, Biometrika, 85, 379-390 (1998) · Zbl 0947.62059
[11] Kavaler, I., Smorodinsky, R., 2019. A cardinal comparison of experts. Mimeo.
[12] Lehrer, E., Any inspection is manipulable, Econometrica, 69, 1333-1347 (2001) · Zbl 1020.91046
[13] Olszewski, W.; Sandroni, A., Manipulability of future-independent tests, Econometrica, 76, 1437-1466 (2008) · Zbl 1154.91604
[14] Pomatto, L., 2016. Testable forecasts. Caltech. Mimeo.
[15] Sandroni, A., The reproducible properties of correct forecasts, Int. J. Game Theory, 32, 151-159 (2003) · Zbl 1071.62084
[16] Sandroni, A.; Smorodinsky, R.; Vohra, R., Calibration with many checking rules, Math. Oper. Res., 28, 141-153 (2003) · Zbl 1082.90544
[17] Shmaya, E., Many inspections are manipulable, Theor. Econ., 3, 367-382 (2008)
[18] Williams, M., Probability with Martingales (1991), Cambridge University Press · Zbl 0722.60001
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.