×

Use and communication of probabilistic forecasts. (English) Zbl 07260607

Summary: Probabilistic forecasts are becoming more and more available. How should they be used and communicated? What are the obstacles to their use in practice? We review experience with five problems where probabilistic forecasting played an important role. This leads us to identify five types of potential users: low stakes users, who do not need probabilistic forecasts; general assessors, who need an overall idea of the uncertainty in the forecast; change assessors, who need to know if a change is out of line with expectations; risk avoiders, who wish to limit the risk of an adverse outcome; and decision theorists, who quantify their loss function and perform the decision-theoretic calculations. This suggests that it is important to interact with users and consider their goals. Cognitive research tells us that calibration is important for trust in probability forecasts and that it is important to match the verbal expression with the task. The cognitive load should be minimized, reducing the probabilistic forecast to a single percentile if appropriate. Probabilities of adverse events and percentiles of the predictive distribution of quantities of interest often seem to be the best way to summarize probabilistic forecasts. Formal decision theory has an important role but in a limited range of applications.

MSC:

62-XX Statistics
68-XX Computer science
PDF BibTeX XML Cite
Full Text: DOI arXiv Link

References:

[1] P. Jorion, Value at Risk: The New Benchmark for Managing Financial Risk (3rd ed.), New York, McGraw-Hill, 2007.
[2] G. E. P. Box, G. M. Jenkins, and G. C. Reinsel, Time Series Analysis, Forecasting and Control (4th ed.), New York, Wiley, 2008. · Zbl 1154.62062
[3] E. W. Steyerberg, Clinical Prediction Models: A Practical Approach to Development, Validation and Updating, New York, Springer, 2009. · Zbl 1314.92010
[4] D. Kahneman, Thinking, Fast and Slow, New York, Farrar, Strauss and Giroux, 2011.
[5] A. E. Raftery, G. H. Givens, and J. E. Zeh, Inference from a deterministic population dynamics model about bowhead whale, balaena mysticetus, replacement yield, Department of Statistics, University of Washington, Technical Report 232, 1992.
[6] A. E. Raftery, G. H. Givens, and J. E. Zeh, Inference from a deterministic population dynamics model for bowhead whales (with discussion), J Am Stat Assoc 90 (1995), 402-430. · Zbl 0925.62473
[7] D. Poole and A. E. Raftery, Inference for deterministic simulation models: the Bayesian melding approach, J Am Stat Assoc 95 (2000), 1244-1255. · Zbl 1072.62544
[8] M. R. Powell, M. Tamplin, B. Marks, and D. T. Compos, Bayesian synthesis of a pathogen growth model: listeria monocytogenes under competition, Int J Food Microb 109 (2006), 34-46.
[9] J. R. Brandon, J. M. Breiwick, A. E. Punt, and P. R. Wade, Constructing a coherent joint prior while respecting biological realism: application to marine mammal stock assessments, ICES J Marine Sci 64 (2007), 1085-1100.
[10] M. G. Falk, R. J. Denham, and K. L. Mengersen, Estimating uncertainty in the revised universal soil loss equation via Bayesian melding, J Agric Biol Environ Stat 15 (2010), 20-37. · Zbl 1306.62270
[11] A. Punt and G. Donovan, Developing management procedures that are robust to uncertainty: lessons from the International Whaling Commission, ICES J Marine Sci 64 (2007), 603-612.
[12] J. G. Cooke, R. Leaper, and V. Papastavrou, Science should not be abandoned in a bid to resolve whaling disputes, Biol Lett 5 (2009), 614-616.
[13] J. G. Cooke, R. Leaper, and V. Papastavrou, Whaling: ways to agree on quotas, Nature 482 (2012), 308.
[14] T. Gneiting and A. E. Raftery, Weather forecasting with ensemble methods, Science 310 (2005), 248-249.
[15] A. E. Raftery, T. Gneiting, F. Balabdaoui, and M. Polakowski, Using Bayesian model averaging to calibrate forecast ensembles, Mon Weather Rev 133 (2005), 1155-1174.
[16] T. Gneiting, A. E. Raftery, A. H. Westveld, and T. Goldman, Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation, Mon Weather Rev 133 (2005), 1098-1118.
[17] J. M. Sloughter, A. E. Raftery, T. Gneiting, and C. Fraley, Probabilistic quantitative precipitation forecasting using Bayesian model averaging, Mon Weather Rev 135 (2007), 3209-3220.
[18] J. M. Sloughter, T. Gneiting, and A. E. Raftery, Probabilistic wind speed forecasting using ensembles and Bayesian model averaging, J Am Stat Assoc 105 (2010), 25-35.
[19] L. Bao, T. Gneiting, E. P. Grimit, P. Guttorp, and A. E. Raftery, Bias correction and Bayesian model averaging for ensemble forecasts of surface wind direction, Mon Weather Rev 138 (2010), 1811-1821.
[20] J. M. Sloughter, T. Gneiting, and A. E. Raftery, Probabilistic wind vector forecasting using ensembles and Bayesian model averaging, Mon Weather Rev 141 (2013), 2107-2119.
[21] R. M. Chmielecki and A. E. Raftery, Probabilistic visibility forecasting using Bayesian model averaging, Mon Weather Rev, 139 (2011), 1626-1636.
[22] C. F. Mass, S. L. Joslyn, J. Pyle, P. Tewson, T. Gneiting, A. E. Raftery, J. Baars, J. M. Sloughter, D. W. Jones, and C. Fraley, PROBCAST: a web-based portal to mesoscale probabilistic forecasts, Bull Am Meteorol Soc 90 (2009), 1009-1014.
[23] S. L. Joslyn and D. W. Jones, Strategies in naturalistic decision-making: a cognitive task analysis of naval weather forecasting. In Naturalistic Decision Making and Macrocognition, J. M. Schraagen, L. Militello, T. Ormerod, and R. Lipshitz, eds. Aldershot, UK, Ashgate Publishing, 2008, 183-202.
[24] S. Joslyn, L. Nadav-Greenberg, and M. U. Taing, The effects of wording on the understanding and use of uncertainty information in a threshold forecasting decision, Appl Cogni Psychol 23 (2008), 55-72.
[25] L. Nadav-Greenberg, S. L. Joslyn, and M. U. Taing, The effect of weather forecast uncertainty visualization on decision making, J Cogni Eng Decis Mak 2 (2008), 24-47.
[26] S. Joslyn, L. Nadav-Greenberg, and R. M. Nichols, Probability of precipitation: assessment and enhancement of end-user understanding, Bull Am Meteorol Soc 90 (2009), 185-193.
[27] S. L. Joslyn and S. Savelli, Communicating forecast uncertainty: public perception of weather forecast uncertainty, Meteorol Appl 17 (2010), 180-195.
[28] D. Kahneman and A. Tversky, Choices, values and frames, Am Psychol 39 (1984), 341-350.
[29] T. Gneiting, F. Balabdaoui, and A. E. Raftery, Probabilistic forecasts, calibration and sharpness, J Roy Stat Soc Ser B 69 (2007), 243-268. · Zbl 1120.62074
[30] S. L. Joslyn, K. Pak, D. W. Jones, J. Pyles, and E. Hunt, The effect of probabilistic information on threshold forecasts, Wea Forecast 22 (2007), 804-812.
[31] L. Nadav-Greenberg and S. L. Joslyn, Uncertainty forecasts improve decision-making among nonexperts, J Cogni Eng Dec Mak 2 (2009), 24-47.
[32] S. L. Joslyn and J. E. LeClerc, Uncertainty forecasts improve weather-related decisions and attenuate the effects of forecast error, J Experim Psychol Appl 18 (2012), 126-140.
[33] K. Fiedler, The dependence of the conjunction fallacy on subtle linguistic factors, Psychol Res 50 (1988), 123-129.
[34] R. Hertwig and G. Gigerenzer, The “conjunction fallacy” revisited: How intelligent inferences look like reasoning errors, J Behav Decis Mak 12 (1999), 275-305.
[35] S. L. Joslyn and R. M. Nichols, Probability or frequency? expressing forecast uncertainty in public weather forecasts, Meterol Appl 16 (2009), 309-314.
[36] D. W. Jones, Probcast: a tool for conveying probabilistic weather information to the public, Presented to the NUOPC Conference, June 2011, 2011.
[37] National Research Council. Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts, Washington, D.C., National Academies Press, 2006.
[38] S. Savelli and S. Joslyn, The advantages of 80
[39] J. von Neumann and O. Morgenstern, Theory of Games and Economic Behavior (2nd ed.), Princeton, NJ, Princeton University Press, 1947. · Zbl 1241.91002
[40] P. Pinson, C. Chevallier, and G. N. Kariniotakis, Trading wind generation with short-term probabilistic forecasts of wind power, IEEE Trans Power Syst 22 (2007), 1148-1156.
[41] P. D. Ghys, T. Brown, N. C. Grassly, G. Garnett, K. A. Stanecki, J. Stover, and N. Walker, The UNAIDS estimation and projection package: a software package to estimate and project national HIV epidemics, Sexual Transm Infect 80 (2004), i5-i9.
[42] UNAIDS, Global Report: UNAIDS Report on the Global AIDS Epidemic, 2013, UNAIDS, Geneva, 2013.
[43] L. Bao, J. A. Salomon, T. Brown, A. E. Raftery, and D. Hogan, Modeling HIV/AIDS epidemics: revised approach in the UNAIDS Estimation and Projection Package 2011, Sexual Transm Infect 88 (2012), i3-i10.
[44] L. Alkema, A. E. Raftery, and S. J. Clark, Probabilistic projections of HIV prevalence using Bayesian melding, Ann Appl Stat 1 (2007), 229-248. · Zbl 1129.62098
[45] T. Brown, J. A. Salomon, L. Alkema, A. E. Raftery, and E. Gouws, Progress and challenges in modelling countrylevel HIV/AIDS epidemics: the UNAIDS Estimation and Projection Package 2007, Sexual Transm Infect 84 (2008), i5-i10.
[46] Intergovernmental Panel on Climate Change. Climate Change 2007: Synthesis Report, IPCC, Geneva, Switzerland, 2007.
[47] K. C. Seto, B. G¨uneral, and L. R. Hutyra, Global forecasts of urban expansion to 2030 and direct impacts on biodiversity and carbon pools, Proc Natl Acad Sci 109 (2012), 16083-16088.
[48] W. Lutz and K. C. Samir, Dimensions of global population projections: what do we know about future population trends and structures? Philosoph Trans Roy Soc B 365 (2010), 2779-2791.
[49] P. K. Whelpton, An empirical method for calculating future population, J Am Stat Assoc 31 (1936), 457-473.
[50] P. H. Leslie, On the use of matrices in certain population dynamics, Biometrika 33 (1945), 183-212. · Zbl 0060.31803
[51] S. H. Preston, P. Heuveline, and M. Guillot, Demography: Measuring and Modeling Population Processes, Malden, MA, Blackwell, 2001.
[52] P. E. Meehl, Clinical versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence, Minneapolis, MN, University of Minesota Press, 1954.
[53] P. E. Meehl, Causes and consequences of my disturbing little book, J Personal Assessm 50 (1986), 370-375.
[54] J. Oeppen and J. W. Vaupel, Broken limits to life expectancy, Science, 296 (2002), 1029-1031.
[55] P. E. Tetlock, Expert Political Judgment: How Good Is It? How Can We Know?, Princeton, NJ, Princeton University Press, 2005.
[56] D. R. Mandel and A. Barnes, Accuracy of forecasts in strategic intelligence, Proc Natl Acad Sci 111 (2014), 10984-10989.
[57] L. Alkema, A. E. Raftery, P. Gerland, S. J. Clark, F. Pelletier, T. Buettner, and G. K. Heilig, Probabilistic projections of the total fertility rate for all countries, Demography 48 (2011), 815-839.
[58] A. E. Raftery, N. Li, H. ˇSevˇcíkov´a, P. Gerland, and G. K. Heilig, Bayesian probabilistic population projections for all countries, Proc Natl Acad Sci 109 (2012), 13915-13921.
[59] A. E. Raftery, J. L. Chunn, P. Gerland, and H. ˇSevˇc´ıkov´a, Bayesian probabilistic projections of life expectancy for all countries, Demography 50 (2013), 777-801.
[60] B. K. Fosdick and A. E. Raftery, Regional probabilistic fertility forecasting by modeling between-country correlations, Demogr Res 30 (2014), 1011-1034.
[61] A. E. Raftery, N. Lalic, and P. Gerland, Joint probabilistic projection of female and male life expectancy, Demogr Res 30 (2014), 795-822.
[62] A. E. Raftery, L. Alkema, and P. Gerland, Bayesian population projections for the United Nations, Stat Sci 29 (2014), 58-68. · Zbl 1332.62428
[63] P. Gerland, A. E. Raftery, H. ˇSevˇc´ıkov‘a, N. Li, D. Gu, T. Spoorenberg, L. Alkema, B. K. Fosdick, J. L. Chunn, N. Lalic, G. Bay, T. Buettner, G. K. Heilig, and J. Wilmoth, World population stabilization unlikely this century, Science 346 (2014), 234-237.
[64] J. Bryant, Probabilistic population forecasts at statistics new zealand: Experience at the national level and plans for the local level, Presented at the Workshop on Use of Probabilistic Forecasts, Royal Statistical Society, London, June 2014, 2014.
[65] P. Gerland, Beyond uncertainty: insights from a demographer about probabilistic forecasts in an international context, Presented at the Workshop on Use of Probabilistic Forecasts, Royal Statistical Society, London, June 2014.
[66] T. A. Louis, Biostatistics and biostatisticians in the policy arena, Presented at the 2012 American Statistical Association Statistics and Public Policy Conference, 2012.
[67] S. Soneji and G. King, Statistical security for social security, Demography 49 (2012), 1037-1060.
[68] D. Kahneman and A. Tversky, Prospect theory: an analysis of decision under risk, Econometrica 47 (1979), 263-291. · Zbl 0411.90012
[69] A. Tversky and D. Kahneman, Advances in prospect theory: cumulative representations of uncertainty, J Risk Uncertain 5 (1992), 297-323. · Zbl 0775.90106
[70] T. Gneiting, Quantiles as optimal point forecasts, Int J Forecast 27 (2011), 197-207.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.