Copula calibration. (English) Zbl 1325.62108

Summary: We propose notions of calibration for probabilistic forecasts of general multivariate quantities. Probabilistic copula calibration is a natural analogue of probabilistic calibration in the univariate setting. It can be assessed empirically by checking for the uniformity of the copula probability integral transform (CopPIT), which is invariant under coordinate permutations and coordinatewise strictly monotone transformations of the predictive distribution and the outcome. The CopPIT histogram can be interpreted as a generalization and variant of the multivariate rank histogram, which has been used to check the calibration of ensemble forecasts. Kendall calibration is an analogue of marginal calibration in the univariate case. Methods and tools are illustrated in simulation studies and applied to compare raw numerical model and statistically postprocessed ensemble forecasts of bivariate wind vectors.


62H05 Characterization and structure theory for multivariate probability distributions; copulas


Full Text: DOI arXiv Euclid


[1] Barbe, P., Genest, C., Ghoudi, K. and Rémillard, B. (1996). On Kendall’s process., J. Multivariate Anal. 58 197-229. · Zbl 0862.60020 · doi:10.1006/jmva.1996.0048
[2] Czado, C., Gneiting, T. and Held, L. (2009). Predictive model assessment for count data., Biometrics 65 1254-1261. · Zbl 1180.62162 · doi:10.1111/j.1541-0420.2009.01191.x
[3] Dawid, A. P. (1984). Statistical theory: The prequential approach., J. R. Stat. Soc. A 147 278-290. · Zbl 0557.62080 · doi:10.2307/2981683
[4] Diebold, F. X., Gunther, T. A. and Tay, A. S. (1998). Evaluating density forecasts with applications to financial risk management., Int. Econ. Rev. 39 863-883.
[5] Diebold, F. X., Hahn, J. and Tay, A. S. (1999). Multivariate density forecast evaluation and calibration in financial risk management: High-frequency returns on foreign exchange., Rev. Econ. Stat. 81 661-673.
[6] Diks, C., Panchenko, V. and van Dijk, D. (2010). Out-of-sample comparisons of copula specifications in multivariate density forecasts., J. Econ. Dyn. Contr. 34 1596-1609. · Zbl 1231.91376 · doi:10.1016/j.jedc.2010.06.021
[7] Eckel, F. A. and Mass, C. F. (2005). Aspects of effective mesoscale, short-range ensemble forecasting., Wea. Forecasting 20 328-350.
[8] Ehm, W. (2011). Unbiased risk estimation and scoring rules., C. R. Math. 349 699-702. · Zbl 1216.62087 · doi:10.1016/j.crma.2011.04.015
[9] Genest, C., Nešlehová, J. and Ziegel, J. (2011). Inference in multivariate Archimedean copula models., Test 20 223-256. · Zbl 1274.62399 · doi:10.1007/s11749-011-0250-6
[10] Gneiting, T., Balabdaoui, F. and Raftery, A. E. (2007). Probabilistic forecasts, calibration and sharpness., J. R. Stat. Soc. B 69 243-268. · Zbl 1120.62074 · doi:10.1111/j.1467-9868.2007.00587.x
[11] Gneiting, T. and Katzfuss, M. (2014). Probabilistic forecasting., Ann. Rev. Stat. Appl. 1 125-151.
[12] Gneiting, T. and Raftery, A. E. (2005). Weather forecasting with ensemble methods., Science 310 248-249.
[13] Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation., J. Am. Stat. Assoc. 102 359-378. · Zbl 1284.62093 · doi:10.1198/016214506000001437
[14] Gneiting, T. and Ranjan, R. (2013). Combining predictive distributions., El. J. Stat. 7 1747-1782. · Zbl 1294.62220 · doi:10.1214/13-EJS823
[15] Gneiting, T., Raftery, A. E., Westveld, A. H. and Goldman, T. (2005). Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation., Mon. Wea. Rev. 133 1098-1118.
[16] Gneiting, T., Stanberry, L. I., Grimit, E. P., Held, L. and Johnson, N. A. (2008). Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds., Test 17 211-235. · Zbl 1196.62091 · doi:10.1007/s11749-008-0114-x
[17] González-Rivera, G. and Yoldas, E. (2012). Autocontour-based evaluation of multivariate predictive densities., Int. J. Forecasting 28 328-342.
[18] Gudendorf, G. and Segers, J. (2010). Extreme-value copulas. In, Copula Theory and Its Applications , (P. Jaworski, F. Durante, W. K. Härdle and T. Rychlik, eds.). Lecture Notes in Statistics - Proceedings 198 127-145. Springer Berlin Heidelberg. · doi:10.1007/978-3-642-12465-5_6
[19] Huber, P. J. (1985). Projection pursuit., Ann. Stat. 13 435-475. · Zbl 0595.62059 · doi:10.1214/aos/1176349519
[20] Ishida, I. (2005). Scanning multivariate conditional densities with probability integral transforms. Center for Advanced Research in Finance, University of Tokio, Working Paper, F-045.
[21] Leutbecher, M. and Palmer, T. N. (2008). Ensemble forecasting., J. Computat. Phys. 227 3515-3539. · Zbl 1132.86308 · doi:10.1016/j.jcp.2007.02.014
[22] McNeil, A. J. and Nešlehová, J. (2009). Multivariate Archimedean copulas, \(d\)-monotone functions and \(l_1\)-norm symmetric distributions., Ann. Stat. 37 3059-3097. · Zbl 1173.62044 · doi:10.1214/07-AOS556
[23] Nelsen, R. B. (2006)., An Introduction to Copulas , 2nd ed. Springer, New York. · Zbl 1152.62030
[24] Nelsen, R. B., Quesada-Molina, J. J., Rodríguez-Lallena, J. A. and Úbeda Flores, M. (2003). Kendall distribution functions., Stat. Prob. Lett. 65 263-268. · Zbl 1048.62058 · doi:10.1016/j.spl.2003.08.002
[25] Pinson, P. (2013). Wind energy: Forecasting challenges for its operational management., Stat. Sci. 28 564-585. · Zbl 1331.91140 · doi:10.1214/13-STS445
[26] Pinson, P. and Girard, R. (2012). Evaluating the quality of scenarios of short-term wind power generation., Appl. Energy 96 12-20.
[27] R Core Team (2013). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria ISBN, 3-900051-07-0.
[28] Röpnack, A., Hense, A., Gebhardt, C. and Majewski, D. (2013). Bayesian model verification of NWP ensemble forecasts., Mon. Wea. Rev. 141 375-387.
[29] Rosenblatt, M. (1952). Remarks on a multivariate transformation., Ann. Math. Stat. 23 470-472. · Zbl 0047.13104 · doi:10.1214/aoms/1177729394
[30] Rüschendorf, L. (2009). On the distributional transform, Sklar’s theorem, and the empirical copula process., J. Stat. Plann. Infer. 139 3921-3927. · Zbl 1171.60313 · doi:10.1016/j.jspi.2009.05.030
[31] Schaake, J., Pailleux, J., Thielen, J., Arritt, R., Hamill, T., Luo, L., Martin, E., McCollor, D. and Pappenberger, F. (2010). Summary of recommendations of the first workshop on Postprocessing and Downscaling Atmospheric Forecasts for Hydrologic Applications held at Météo-France, Toulouse, France, 15-18 June 2009., Atmos. Sci. Let. 11 59-63.
[32] Schefzik, R., Thorarinsdottir, T. L. and Gneiting, T. (2013). Uncertainty quantification in complex simulation models using ensemble copula coupling., Stat. Sci. 28 616-640. · Zbl 1331.62265 · doi:10.1214/13-STS443
[33] Schuhen, N., Thorarinsdottir, T. L. and Gneiting, T. (2012). Ensemble model output statistics for wind vectors., Mon. Wea. Rev. 140 3204-3219.
[34] Smith, L. A. and Hansen, J. A. (2004). Extending the limits of ensemble forecast verification with the minimum spanning tree histogram., Mon. Wea. Rev. 132 1522-1528.
[35] Tawn, J. A. (1988). Bivariate extreme value theory: Models and estimation., Biometrika 75 397-415. · Zbl 0653.62045 · doi:10.1093/biomet/75.3.397
[36] Thorarinsdottir, T. L., Scheuerer, M. and Heinz, C. (2014). Assessing the calibration of high-dimensional ensemble forecasts using the rank histogram., J. Comput. Graph. Stat. ,
[37] Wilks, D. S. (2004). The minimum spanning tree histogram as a verification tool for multidimensional ensemble forecasts., Mon. Wea. Rev. 132 1329-1340.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.