Local proper scoring rules of order two. (English) Zbl 1246.86013

Summary: Scoring rules assess the quality of probabilistic forecasts, by assigning a numerical score based on the predictive distribution and on the event or value that materializes. A scoring rule is proper if it encourages truthful reporting. It is local of order \(k\) if the score depends on the predictive density only through its value and the values of its derivatives of order up to \(k\) at the realizing event. Complementing fundamental recent work by Parry, Dawid and Lauritzen, we characterize the local proper scoring rules of order 2 relative to a broad class of Lebesgue densities on the real line, using a different approach. In a data example, we use local and nonlocal proper scoring rules to assess statistically postprocessed ensemble weather forecasts.


86A32 Geostatistics
62C99 Statistical decision theory
62M20 Inference from stochastic processes and prediction
86A10 Meteorology and atmospheric physics
Full Text: DOI arXiv Euclid


[1] Bauer, H. (2001). Measure and Integration Theory. de Gruyter Studies in Mathematics 26 . de Gruyter, Berlin. · Zbl 0985.28001
[2] Bernardo, J.-M. (1979). Expected information as expected utility. Ann. Statist. 7 686-690. · Zbl 0407.62002 · doi:10.1214/aos/1176344689
[3] Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review 78 1-3.
[4] Bröcker, J. and Smith, L. A. (2008). From ensemble forecasts to predictive distribution functions. Tellus Ser. A 60 663-678.
[5] DasGupta, A. (2008). Asymptotic Theory of Statistics and Probability . Springer, New York. · Zbl 1154.62001
[6] Dawid, A. P. (1984). Statistical theory. The prequential approach. J. Roy. Statist. Soc. Ser. A 147 278-292. · Zbl 0557.62080 · doi:10.2307/2981683
[7] Dawid, A. P. (2007). The geometry of proper scoring rules. Ann. Inst. Statist. Math. 59 77-93. · Zbl 1108.62009 · doi:10.1007/s10463-006-0099-8
[8] Dawid, A. P. (2008). Comments on: Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds [MR2434318]. TEST 17 243-244. · Zbl 1367.62202 · doi:10.1007/s11749-008-0118-6
[9] Dawid, A. P. and Lauritzen, S. L. (2005). The geometry of decision theory. In Proceedings of the Second International Symposium on Information Geometry and Its Applications 22-28. Univ. Tokyo, Tokyo, Japan.
[10] Dawid, A. P., Lauritzen, S. and Parry, M. (2012). Proper local scoring rules on discrete sample spaces. Ann. Statist. 40 593-608. · Zbl 1246.62010
[11] Dawid, A. P., Parry, M. and Lauritzen, S. (2009). Personal communication.
[12] Ehm, W. (2011). Unbiased risk estimation and scoring rules. C. R. Math. Acad. Sci. Paris 349 699-702. · Zbl 1216.62087 · doi:10.1016/j.crma.2011.04.015
[13] Ehm, W. and Gneiting, T. (2009). Local proper scoring rules. Technical Report 551, Dept. Statistics, Univ. Washington. (Addendum 2010). · Zbl 1246.86013
[14] Gelfand, I. M. and Fomin, S. V. (1963). Calculus of Variations . Prentice Hall International, Englewood Cliffs, NJ. · Zbl 0127.05402
[15] Genton, M. G., ed. (2004). Skew-elliptical Distributions and Their Applications : A Journey Beyond Normality . Chapman & Hall/CRC, Boca Raton, FL. · Zbl 1069.62045
[16] Gneiting, T. (2008). Editorial: Probabilistic forecasting. J. Roy. Statist. Soc. Ser. A 171 319-321. · doi:10.1111/j.1467-985X.2007.00522.x
[17] Gneiting, T., Balabdaoui, F. and Raftery, A. E. (2007). Probabilistic forecasts, calibration and sharpness. J. R. Stat. Soc. Ser. B Stat. Methodol. 69 243-268. · Zbl 1120.62074 · doi:10.1111/j.1467-9868.2007.00587.x
[18] Gneiting, T. and Raftery, A. E. (2005). Atmospheric science. Weather forecasting with ensemble methods. Science 310 248-249.
[19] Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102 359-378. · Zbl 1284.62093 · doi:10.1198/016214506000001437
[20] Gneiting, T., Raftery, A. E., Westveld, A. H. and Goldman, T. (2005). Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Monthly Weather Review 133 1098-1118.
[21] Good, I. J. (1952). Rational decisions. J. Roy. Statist. Soc. Ser. B. 14 107-114.
[22] Grimit, E. P. and Mass, C. F. (2002). Initial results of a mesoscale short-range ensemble system over the Pacific Northwest. Weather and Forecasting 17 192-205.
[23] Hendrickson, A. D. and Buehler, R. J. (1971). Proper scores for probability forecasters. Ann. Math. Statist. 42 1916-1921. · Zbl 0231.62018 · doi:10.1214/aoms/1177693057
[24] Huber, P. J. (1974). Fisher information and spline interpolation. Ann. Statist. 2 1029-1033. · Zbl 0289.62032 · doi:10.1214/aos/1176342822
[25] Hyvärinen, A. (2005). Estimation of non-normalized statistical models by score matching. J. Mach. Learn. Res. 6 695-709 (electronic). · Zbl 1222.62051
[26] Hyvärinen, A. (2007). Some extensions of score matching. Comput. Statist. Data Anal. 51 2499-2512. · Zbl 1161.62326
[27] Jose, V. R. R., Nau, R. F. and Winkler, R. L. (2009). Sensitivity to distance and baseline distributions in forecast evaluation. Management Science 55 582-590.
[28] Mason, S. J. (2008). Understanding forecast verification statistics. Meteorological Applications 15 31-40.
[29] Matheson, J. E. and Winkler, R. L. (1976). Scoring rules for continuous probability distributions. Management Science 22 1087-1096. · Zbl 0349.62080 · doi:10.1287/mnsc.22.10.1087
[30] Palmer, T. N. (2002). The economic value of ensemble forecasts as a tool for risk assessment: From days to decades. Quarterly Journal of the Royal Meteorological Society 128 747-774.
[31] Parry, M., Dawid, A. P. and Lauritzen, S. (2012). Proper local scoring rules. Ann. Statist. 40 561-592. · Zbl 1246.62011
[32] Raftery, A. E., Gneiting, T., Balabdaoui, F. and Polakowski, M. (2005). Using Bayesian model averaging to calibrate forecast ensembles. Monthly Weather Review 133 1155-1174.
[33] Sloughter, M., Gneiting, T. and Raftery, A. E. (2010). Probabilistic wind spread forecasting using ensembles and Bayesian model averaging. J. Amer. Statist. Assoc. 105 25-35. · doi:10.1198/jasa.2009.ap08615
[34] Sloughter, J. M., Raftery, A. E., Gneiting, T. and Fraley, C. (2007). Probabilistic quantitative precipitation forecasting using Bayesian model averaging. Monthly Weather Review 135 3209-3220.
[35] Staël von Holstein, C. A. S. (1969). A family of strictly proper scoring rules which are sensitive to distance. Journal of Applied Meteorology 9 360-364.
[36] Thorarinsdottir, T. L. and Gneiting, T. (2010). Probabilistic forecasts of wind speed: Ensemble model ouput statistics by using heteroscedastic censored regression. J. Roy. Statist. Soc. Ser. A 173 371-388. · doi:10.1111/j.1467-985X.2009.00616.x
[37] Villani, C. (2009). Optimal Transport : Old and New. Grundlehren der Mathematischen Wissenschaften [ Fundamental Principles of Mathematical Sciences ] 338 . Springer, Berlin. · Zbl 1156.53003
[38] Wilks, D. S. and Hamill, T. M. (2007). Comparison of ensemble-MOS methods using GFS reforecasts. Monthly Weather Review 135 2379-2390.
[39] Winkler, R. L. and Jose, V. R. R. (2008). Comments on: Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds [MR2434318]. TEST 17 251-255. · Zbl 1367.62207 · doi:10.1007/s11749-008-0121-y
[40] Winkler, R. L. and Murphy, A. H. (1968). “Good” probability assessors. Journal of Applied Meteorology 7 751-758.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.