Proper local scoring rules. (English) Zbl 1246.62011

Parry, Matthew; Dawid, A. Philip; Lauritzen, Steffen

Ann. Stat. 40, No. 1, 561-592 (2012).

Summary: We investigate proper scoring rules for continuous distributions on the real line. It is known that the log score is the only such rule that depends on the quoted density only through its value at the outcome that materializes. Here we allow further dependence on a finite number \(m\) of derivatives of the density at the outcome, and describe a large class of such \(m-local\) proper scoring rules: these exist for all even \(m\) but no odd \(m\). We further show that for \(m \geq 2\) all such \(m\)-local rules can be computed without knowledge of the normalizing constant of the distribution.

MSC:

62C99	Statistical decision theory
62A99	Foundational topics in statistics

Keywords:

Bregman score; concavity; divergence; entropy; Euler-Lagrange equation; homogeneity; integration by parts; local function; score matching; variational methods

References:

[1]	Barndorff-Nielsen, O. (1976). Plausibility inference. J. Roy. Statist. Soc. Ser. B 38 103-131. · Zbl 0403.62008
[2]	Bernardo, J.-M. (1979). Expected information as expected utility. Ann. Statist. 7 686-690. · Zbl 0407.62002 · doi:10.1214/aos/1176344689
[3]	Bregman, L. M. (1967). The relaxation method of finding a common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. and Math. Phys. 7 200-217. · Zbl 0186.23807
[4]	Csiszár, I. (1991). Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems. Ann. Statist. 19 2032-2066. · Zbl 0753.62003 · doi:10.1214/aos/1176348385
[5]	Dawid, A. P. (1986). Probability forecasting. In Encyclopedia of Statistical Sciences (S. Kotz, N. L. Johnson and C. B. Read, eds.) 7 210-218. Wiley, New York.
[6]	Dawid, A. P. (1998). Coherent measures of discrepancy, uncertainty and dependence, with applications to Bayesian predictive experimental design. Technical Report 139, Dept. Statistical Science, Univ. College London. Available at .
[7]	Dawid, A. P. (2007). The geometry of proper scoring rules. Ann. Inst. Statist. Math. 59 77-93. · Zbl 1108.62009 · doi:10.1007/s10463-006-0099-8
[8]	Dawid, A. P. and Lauritzen, S. L. (2005). The geometry of decision theory. In Proceedings of the Second International Symposium on Information Geometry and Its Applications 22-28. Univ. Tokyo, Tokyo, Japan.
[9]	Dawid, A. P., Lauritzen, S. and Parry, M. (2012). Proper local scoring rules on discrete sample spaces. Ann. Statist. 40 593-608. · Zbl 1246.62010
[10]	Eguchi, S. (2008). Information divergence geometry and the application to statistical machine learning. In Information Theory and Statistical Learning (F. Emmert-Streib and M. Dehmer, eds.) 309-332. Springer, New York. · Zbl 1183.68473
[11]	Ehm, W. and Gneiting, T. (2010). Local proper scoring rules. Technical Report 551, Dept. Statistics, Univ. Washington. Available at . · Zbl 1246.86013
[12]	Ehm, W. and Gneiting, T. (2012). Local proper scoring rules of order two. Ann. Statist. 40 609-637. · Zbl 1246.86013
[13]	Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J. and Stahel, W. A. (1986). Robust Statistics : The Approach Based on Influence Functions . Wiley, New York. · Zbl 0593.62027
[14]	Huber, P. J. (1981). Robust Statistics . Wiley, New York. · Zbl 0536.62025
[15]	Hyvärinen, A. (2005). Estimation of non-normalized statistical models by score matching. J. Mach. Learn. Res. 6 695-709 (electronic). · Zbl 1222.62051
[16]	Hyvärinen, A. (2007). Some extensions of score matching. Comput. Statist. Data Anal. 51 2499-2512. · Zbl 1161.62326
[17]	Savage, L. J. (1954). The Foundations of Statistics . Wiley, New York. · Zbl 0055.12604
[18]	Troutman, J. L. (1983). Variational Calculus with Elementary Convexity . Springer, New York. · Zbl 0523.49001
[19]	van Brunt, B. (2004). The Calculus of Variations . Springer, New York. · Zbl 1039.49001