×

zbMATH — the first resource for mathematics

Combining predictive distributions. (English) Zbl 1294.62220
Summary: In probabilistic forecasting combination formulas for the aggregation of predictive distributions need to be estimated based on past experience and training data. We study combination formulas and aggregation methods for predictive cumulative distribution functions from the perspectives of calibration and dispersion, taking an original prediction space approach that applies to discrete, mixed discrete-continuous and continuous predictive distributions alike. The key idea is that aggregation methods ought to be parsimonious, yet sufficiently flexible to accommodate any type of dispersion in the component distributions. Both linear and non-linear aggregation methods are investigated, including generalized, spread-adjusted and beta-transformed linear pools. The effects and techniques are demonstrated theoretically, in simulation examples, and in case studies, where we fit combination formulas for density forecasts of S&P 500 returns and daily maximum temperature at Seattle-Tacoma Airport.

MSC:
62M20 Inference from stochastic processes and prediction
62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62P05 Applications of statistics to actuarial sciences and financial mathematics
62P12 Applications of statistics to environmental and related topics
91B06 Decision theory
91G80 Financial applications of other theories
Software:
fGarch; RMetrics; R
PDF BibTeX XML Cite
Full Text: DOI Euclid arXiv
References:
[1] Allard, A., Comunian, A. and Renard, P. (2012). Probability aggregation methods in geoscience. Mathematical Geosciences , 44 , 545-581. · Zbl 1256.86006
[2] Arnold, H. M., Moroz, I. M. and Palmer, T. N. (2013). Stochastic parameterizations and model uncertainty in the Lorenz ’96 system. Philiosophical Transactions of the Royal Society Ser. A , 371 , 20110479.
[3] Berrocal, V. J., Raftery, A. E. and Gneiting, T. (2007). Combining spatial statistical and ensemble information for probabilistic weather forecasting. Monthly Weather Review , 135 , 1386-1402.
[4] Bollerslev, T. (1986). Generalized autoregressive conditional heteroscedasticity, Journal of Econometrics , 31 , 307-327. · Zbl 0616.62119
[5] Bröcker, J. (2009). Reliability, sufficiency, and the decomposition of proper scores. Quarterly Journal of the Royal Meteorological Society , 135 , 1512-1519.
[6] Clemen, R. T. and Winkler, R. L. (2007). Aggregating probability distributions. In Ward, E., Miles, R. F. and von Winterfeldt, D. (eds.), Advances in Decision Analysis: From Foundations to Applications , Cambridge University Press, pp. 154-176.
[7] Clements, M. P. and Harvey, D. I. (2011). Combining probability forecasts. International Journal of Forecasting , 27 , 208-223.
[8] Cox, D. R. (1958). Two further applications of a model for binary regression. Biometrika , 45 , 562-565. · Zbl 0085.13715
[9] Czado, C., Gneiting, T. and Held, L. (2009). Predictive model assessment for count data. Biometrics , 65 , 1254-1261. · Zbl 1180.62162
[10] Dawid, A. P. (1984). Statistical theory: The prequential approach (with discussion and rejoinder). Journal of the Royal Statistical Society Ser. A , 147 , 278-290. · Zbl 0557.62080
[11] Dawid, A. P. (1986). Probability forecasting, in Kotz, S., Johnson, N. L. and Read, C. B. (eds.), Encyclopedia of Statistical Sciences , Vol. 7, Wiley, New York, pp. 210-218.
[12] Dawid, A. P., DeGroot, M. H. and Mortera, J. (1995). Coherent combination of experts’ opinions (with discussion and rejoinder). Test , 4 , 263-313. · Zbl 0844.62002
[13] DelSole, T. and Shukla, J. (2010). Model fidelity versus skill in seasonal forecasting. Journal of Climate , 23 , 4794-4806.
[14] Diebold, F. X., Gunther, T. A. and Tay, A. S. (1998). Evaluating density forecasts with applications to financial risk management. International Economic Review , 39 , 863-883.
[15] Dutton, J. A. (2002). Opportunities and priorities in a new era for weather and climate services. Bulletin of the American Meteorological Society , 83 , 1303-1311.
[16] Eckel, F. A. and Mass, C. F. (2005). Aspects of effective short-range ensemble forecasting. Weather and Forecasting , 20 , 328-350.
[17] Ferguson, T. S. (1996). A Course in Large Sample Theory . Chapman and Hall. · Zbl 0871.62002
[18] Fraley, C., Raftery, A. E., Gneiting, T., Sloughter, J. M. and Berrocal, V. J. (2011). Probabilistic weather forecasting in R. R Journal , 3(1) , 55-63.
[19] French, S. and Ríos Insua, D. (2000). Statistical Decision Theory . Arnold, London.
[20] Fricker, T. E., Ferro, C. A. T. and Stephenson, D. B. (2013). Three recommendations for evaluating climate predictions. Meteorological Applications , 20 , 246-255.
[21] Garratt, A., Mitchell, J., Vahey, S. P. and Wakerly, E. C. (2011). Real-time inflation forecast densities from ensemble Phillips curves. North American Journal of Economics and Finance , 22 , 77-87.
[22] Genest, C. (1984a). A characterization theorem for externally Bayesian groups. Annals of Statistics , 12 , 1100-1105. · Zbl 0541.62003
[23] Genest, C. (1984b). A conflict between two axioms for combining subjective distributions. Journal of the Royal Statistical Society Ser. B , 46 , 403-405. · Zbl 0567.62005
[24] Genest, C. and Zidek, J. (1986). Combining probability distributions: A critique and an annotated bibliography. Statistical Science , 1 , 114-135. · Zbl 0587.62017
[25] Genest, C., McConway, K. J. and Schervish, M. J. (1986). Characterization of externally Bayesian pooling operators. Annals of Statistics , 14 , 487-501. · Zbl 0602.62005
[26] Geweke, J. and Amisano, G. (2011). Optimal prediction pools. Journal of Econometrics , 164 , 130-141. · Zbl 1441.62700
[27] Glahn, B., Peroutka, M., Wiedenfeld, J., Wagner, J., Zylstra, G., Schuknecht, B. and Jackson, B. (2009). MOS uncertainty estimates in an ensemble framework. Monthly Weather Review , 137 , 246-268.
[28] Gneiting, T. (2008). Editorial: Probabilistic forecasting. Journal of the Royal Statistical Society Ser. A , 171 , 319-321.
[29] Gneiting, T. and Raftery, A. E. (2005). Weather forecasting with ensemble methods. Science , 310 , 248-249.
[30] Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association , 102 , 359-378. · Zbl 1284.62093
[31] Gneiting, T., Balabdaoui, F. and Raftery, A. E. (2007). Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society Ser. B , 69 , 243-268. · Zbl 1120.62074
[32] Hall, S. G. and Mitchell, J. (2007). Combining density forecasts. International Journal of Forecasting , 23 , 1-13.
[33] Hamill, T. M. (2001). Interpretation of rank histograms for verifying ensemble forecasts. Monthly Weather Review , 129 , 550-560.
[34] Held, L., Rufibach, K. and Balabdaoui, F. (2010). A score regression approach to assess calibration of continuous probabilistic predictions. Biometrics , 66 , 1295-1305. · Zbl 1208.62077
[35] Hora, S. C. (2004). Probability judgements for continuous quantities: Linear combinations and calibration. Management Science , 50 , 597-604.
[36] Hora, S. C. (2010). An analytic method for evaluating the performance of aggregation rules for probability densities. Operations Research , 58 , 1440-1449. · Zbl 1228.91022
[37] Hoeting, J. A., Madigan, D., Raftery, A. E. and Volinsky, C. T. (1999) Bayesian model averaging: A tutorial (with discussion). Statistal Science , 4 , 382-417. · Zbl 1059.62525
[38] Jore, A. S., Mitchell, J. and Vahey, S. P. (2010). Combining forecast densities from VARs with uncertain stabilities. Journal of Applied Econometrics , 25 , 621-634.
[39] Jouini, M. N. and Clemen, R. T. (1996). Copula models for aggregating expert opinions. Operations Research , 44 , 444-457. · Zbl 0864.90067
[40] Kascha, C. and Ravazzolo, F. (2010). Combining inflation density forecasts. Journal of Forecasting , 29 , 231-250. · Zbl 1204.91092
[41] Kleiber, W., Raftery, A. E., Baars, J., Gneiting, T., Mass, C. F. and Grimit, E. (2011). Locally calibrated probabilistic temperature forecasting using geostatistical model averaging and local Bayesian model averaging. Monthly Weather Review , 139 , 2630-2649.
[42] Krüger, F. (2013). Jensen’s inequality and the success of linear prediction pools. Discussion paper, .
[43] Luss, R., Rosset, S. and Shahar, M. (2012). Efficient regularized isotonic regression with application to gene-gene interaction search. Annals of Applied Statistics , 6 , 253-283. · Zbl 1235.62046
[44] Madigan, D. and Raftery, A. E. (1994). Model selection and accounting for model uncertainty in graphical models using Occam’s window. Journal of the American Statistical Association , 89 , 1535-1546. · Zbl 0814.62030
[45] Mason, S. J., Galpin, S., Goddard, L., Graham, N. E. and Rajaratnam, B. (2007). Conditional exceedance probabilities. Monthly Weather Review , 135 , 363-372.
[46] McConway, K. J. (1981). Marginalization and linear opinion pools. Journal of the American Statistical Association , 76 , 410-414. · Zbl 0455.90004
[47] Mitchell, J. and Hall, S. G. (2005). Evaluating, comparing and combining density forecasts using the KLIC with an application to the Bank of England and NIESR ‘fan’ charts of inflation. Oxford Bulletin of Economics and Statistics , 67 , 995-1033.
[48] Murphy, A. H. and Winkler, R. L. (1987). A general framework for forecast verification. Monthly Weather Review , 115 , 1330-1338.
[49] Murphy, A. H. and Winkler, R. L. (1992). Diagnostic verification of probability forecasts. International Journal of Forecasting , 7 , 435-455.
[50] Platt, J. C. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Smola, A., Bartlett, P., Schölkopf, B. and Schuurmans, D., eds., Advances in Large Margins Classifiers , MIT Press, pp. 61-74.
[51] Primo, C., Ferro, C. A. T., Jolliffe, I. T. and Stephenson, D. B. (2009). Calibration of probabilistic forecasts of binary events. Monthly Weather Review , 137 , 1142-1149.
[52] Raftery, A. E., Gneiting, T., Balabdaoui, F. and Polakowski, M. (2005). Using Bayesian model averaging to calibrate forecast ensembles. Monthly Weather Review , 133 , 1155-1174.
[53] Ranjan, R. and Gneiting, T. (2010). Combining probability forecasts. Journal of the Royal Statistical Society Ser. B , 72 , 71-91.
[54] R Development Core Team (2011). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0, .
[55] Rosenblatt, M. (1952). Remarks on a multivariate transformation. Annals of Mathematical Statistics , 23 , 470-472. · Zbl 0047.13104
[56] Rüschendorf, L. (1981). Stochastically ordered distributions and monotonicity of the OC-function of sequential probability ratio tests. Mathematische Operationsforschung Ser. Statistics , 12 , 327-338. · Zbl 0481.62063
[57] Schervish, M. J. (1989). A general method for comparing probability assessors. Annals of Statistics , 17 , 1856-1879. · Zbl 0703.62008
[58] Sloughter, J. M., Raftery, A. E., Gneiting, T. and Fraley, C. (2007). Probabilistic quantitative precipitation forecasting using Bayesian model averaging. Monthly Weather Review , 135 , 3209-3220.
[59] Stone, M. (1961). The linear pool. Annals of Mathematical Statistics , 32 , 1339-1342. · Zbl 0109.38004
[60] Tsyplakov, A. (2011). Evaluating density forecasts: a comment. MPRA paper no. 31233, .
[61] Tsyplakov, A. (2013). Evaluation of probabilistic forecasts: proper scoring rules and moments. Preprint, .
[62] Wallis, K. F. (2005). Combining density and interval forecasts: A modest proposal. Oxford Bulletin of Economics and Statistics , 67 , 983-994.
[63] Winkler, R. L. (1968). The consensus of subjective probability distributions. Management Science , 15 , B61-B75.
[64] Winkler, R. L. (1986). Comment on “Combining probability distributions: A critique and an annotated bibliography”. Statistical Science , 1 , 138-140. · Zbl 0587.62017
[65] Wuertz, D. and Rmetrics Core Team (2007). The fGarch Package. Reference manual, available at Garch.pdf .
[66] Zadrozny, B. and Elkan, C. (2002). Transforming classifier scores into accurate multiclass probability estimates. Proceedings of the eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 694-699.
[67] Zarnowitz, V. (1969). The new ASA-NBER survey of forecasts by economic statisticians. American Statistician , 23 , 12-16.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.