Higher order elicitability and Osband’s principle. (English) Zbl 1355.62006

Ann. Stat. 44, No. 4, 1680-1707 (2016); correction ibid. 49, No. 1, 614 (2021).
The article contributes to the decision-theoretic framework for the evaluation of point forecasts. Let \(Y \in \mathbb{R}^d\) be a random variable observed by a forecaster, \(F\) be its cumulative distribution function, \(T(F) \in \mathbb{R}^k\) be a functional and \(S(x,y)\) be a scoring function (\(x \in \mathbb{R}^k\), \(y \in \mathbb{R}^d\)). A scoring function is said to be strictly consistent for \(T(F)\) if \(x=T(F)\) is the unique minimizer for \(E_F(S(x,Y))\) for all \(F\). A functional \(T(F)\) is called elicitable if there exists a strictly consistent scoring function for it. In case \(d=k=1\), the examples are moments, ratios of moments, quantiles and expectiles. However, the variance is not an elicitable functional if \(k=1\) but may be a component of an elicitable functional if \(k \geq 2\).
In the paper, the necessary and sufficient conditions for strictly consistent scoring functions are given. Some new examples of one-dimensional functionals which are not elicitable but may be a component of multi-dimensional elicitable functional are considered.


62C05 General considerations in statistical decision theory
62C20 Minimax procedures in statistical decision theory
62P05 Applications of statistics to actuarial sciences and financial mathematics
91B06 Decision theory
Full Text: DOI arXiv Euclid


[1] Abernethy, J. D. and Frongillo, R. M. (2012). A characterization of scoring rules for linear properties. In Proceedings of the 25 th Annual Conference on Learning Theory. Journal of Machine Learning Research : Workshop and Conference Proceedings , Vol. 23 (S. Mannor, N. Srebro andR. C. Williamson, eds.) 27.1-27.13.
[2] Acerbi, C. (2002). Spectral measures of risk: A coherent representation of subjective risk aversion. J. Bank. Financ. 26 1505-1518.
[3] Acerbi, C. and Székely, B. (2014). Backtesting expected shortfall. Risk Mag. 27 76-81.
[4] Artzner, P., Delbaen, F., Eber, J.-M. and Heath, D. (1999). Coherent measures of risk. Math. Finance 9 203-228. · Zbl 0980.91042
[5] Banerjee, A., Guo, X. and Wang, H. (2005). On the optimality of conditional expectation as a Bregman predictor. IEEE Trans. Inform. Theory 51 2664-2669. · Zbl 1284.94025
[6] Bellini, F. and Bignozzi, V. (2015). On elicitable risk measures. Quant. Finance 15 725-733. · Zbl 1395.91506
[7] Cont, R., Deguest, R. and Scandolo, G. (2010). Robustness and sensitivity analysis of risk measurement procedures. Quant. Finance 10 593-606. · Zbl 1192.91191
[8] Daníelsson, J., Embrechts, P., Goodhart, C., Keating, C., Muennich, F., Renault, O. and Shin, H. S. (2001). An academic response to Basel II. Special Paper No. 130.
[9] Davis, M. H. A. (2016). Verification of internal risk measure estimates. Stat. Risk Model . · Zbl 1356.91102
[10] Dawid, A. P. and Sebastiani, P. (1999). Coherent dispersion criteria for optimal experimental design. Ann. Statist. 27 65-81. · Zbl 0948.62057
[11] Delbaen, F. (2012). Monetary Utility Functions . Osaka Univ. Press, Osaka.
[12] Delbaen, F., Bellini, F., Bignozzi, V. and Ziegel, J. F. (2016). Risk measures with the CxLS property. Finance Stoch. 20 433-453. · Zbl 1376.91173
[13] Embrechts, P. and Hofert, M. (2014). Statistics and quantitative risk management for banking and insurance. Ann. Rev. Stat. Appl. 1 .
[14] Emmer, S., Kratz, M. and Tasche, D. (2015). What is the best risk measure in practice? A comparison of standard measures. J. Risk 18 31-60.
[15] Engelberg, J., Manski, C. F. and Williams, J. (2009). Comparing the point predictions and subjective probability distributions of professional forecasters. J. Bus. Econom. Statist. 27 30-41.
[16] Fissler, T. and Ziegel, J. F. (2016). Supplement to “Higher order elicitability and Osband’s principle.” . · Zbl 1355.62006
[17] Frongillo, R. and Kash, I. (2015). Vector-valued property elicitation. J. Mach. Learn. Res. Workshop Conf. Proc. 40 1-18.
[18] Gneiting, T. (2011). Making and evaluating point forecasts. J. Amer. Statist. Assoc. 106 746-762. · Zbl 1232.62028
[19] Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102 359-378. · Zbl 1284.62093
[20] Grauert, H. and Fischer, W. (1978). Differential- und Integralrechnung. II : Differentialrechnung in Mehreren Veränderlichen Differentialgleichungen , corrected ed. Heidelberger Taschenbücher [ Heidelberg Paperbacks ] 36 . Springer, Berlin. · Zbl 0364.26002
[21] Heinrich, C. (2014). The mode functional is not elicitable. Biometrika 101 245-251. · Zbl 1400.62026
[22] Huber, P. J. (1964). Robust estimation of a location parameter. Ann. Math. Statist. 35 73-101. · Zbl 0136.39805
[23] Jouini, E., Schachermayer, W. and Touzi, N. (2006). Law invariant risk measures have the Fatou property. In Advances in Mathematical Economics. Vol. 9. Adv. Math. Econ. 9 49-71. Springer, Tokyo. · Zbl 1198.46028
[24] Koenker, R. (2005). Quantile Regression. Econometric Society Monographs 38 . Cambridge Univ. Press, Cambridge. · Zbl 1111.62037
[25] Kou, S. and Peng, X. (2014). On the measurement of economic tail risk. Preprint. Available at . · Zbl 1378.91101
[26] Kou, S., Peng, X. and Heyde, C. C. (2013). External risk measures and Basel accords. Math. Oper. Res. 38 393-417. · Zbl 1297.91089
[27] Krätschmer, V., Schied, A. and Zähle, H. (2012). Qualitative and infinitesimal robustness of tail-dependent statistical functionals. J. Multivariate Anal. 103 35-47. · Zbl 1236.62043
[28] Krätschmer, V., Schied, A. and Zähle, H. (2015). Quasi-Hadamard differentiability of general risk functionals and its applications. Stat. Risk Model. 32 25-47. · Zbl 1346.60019
[29] Krätschmer, V., Schied, A. and Zähle, H. (2014). Comparative and qualitative robustness for law-invariant risk measures. Finance Stoch. 18 271-295. · Zbl 1298.91195
[30] Kusuoka, S. (2001). On law invariant coherent risk measures. In Advances in Mathematical Economics , Vol. 3. Adv. Math. Econ. 3 83-95. Springer, Tokyo. · Zbl 1010.60030
[31] Lambert, N. (2013). Elicitation and evaluation of statistical functionals. Preprint. Available at .
[32] Lambert, N., Pennock, D. M. and Shoham, Y. (2008). Eliciting properties of probability distributions. In Proceedings of the 9 th ACM Conference on Electronic Commerce 129-138. ACM, Chicago, IL.
[33] Murphy, A. H. and Daan, H. (1985). Forecast evaluation. In Probability , Statistics and Decision Making in the Atmospheric Sciences (A. H. Murphy and R. W. Katz, eds.) 379-437. Westview Press, Boulder, CO.
[34] Newey, W. K. and Powell, J. L. (1987). Asymmetric least squares estimation and testing. Econometrica 55 819-847. · Zbl 0625.62047
[35] Osband, K. H. (1985). Providing incentives for better cost forecasting. Ph.D. thesis, Univ. California, Berkeley.
[36] Osband, K. and Reichelstein, S. (1985). Information-eliciting compensation schemes. J. Public Econ. 27 107-115.
[37] Savage, L. J. (1971). Elicitation of personal probabilities and expectations. J. Amer. Statist. Assoc. 66 783-801. · Zbl 0253.92008
[38] Steinwart, I., Pasin, C., Williamson, R. and Zhang, S. (2014). Elicitation and identification of properties. In Proceedings of the 27 th Conference on Learning Theory. Journal of Machine Learning Research : Workshop and Conference Proceedings , Vol. 35 (M. F. Balcan andC. Szepesvari, eds.) 482-526.
[39] Wang, R. and Ziegel, J. F. (2015). Elicitable distortion risk measures: A concise proof. Statist. Probab. Lett. 100 172-175. · Zbl 1320.91097
[40] Weber, S. (2006). Distribution-invariant risk measures, information, and dynamic consistency. Math. Finance 16 419-441. · Zbl 1145.91037
[41] Ziegel, J. F. (2014). Coherence and elicitability. Math. Finance . · Zbl 1390.91336
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.