The complexity of forecast testing.

*(English)*Zbl 1160.91396Summary: Consider a weather forecaster predicting a probability of rain for the next day. We consider tests that, given a finite sequence of forecast predictions and outcomes, will either pass or fail the forecaster. Sandroni showed that any test which passes a forecaster who knows the distribution of nature can also be probabilistically passed by a forecaster with no knowledge of future events. We look at the computational complexity of such forecasters and exhibit a linear-time test and distribution of nature such that any forecaster without knowledge of the future who can fool the test must be able to solve computationally difficult problems. Thus, unlike Sandroni’s work, a computationally efficient forecaster cannot always fool this test independently of nature.

##### MSC:

91B82 | Statistical methods; economic indices and measures |

62M09 | Non-Markovian processes: estimation |

60G25 | Prediction theory (aspects of stochastic processes) |

91A26 | Rationality and learning in game theory |

62G10 | Nonparametric hypothesis testing |

62P30 | Applications of statistics in engineering and industry; control charts |