Fisher’s information and the class of \(f\)-divergences based on extended Arimoto’s entropies. (English) Zbl 1397.62028

Summary: I. Csiszár [Publ. Math. Inst. Hung. Acad. Sci., Ser. A 8, 85–108 (1963; Zbl 0124.08703)] and S. M. Ali and S. D. Silvey [J. R. Stat. Soc., Ser. B 28, 131–142 (1966; Zbl 0203.19902)] introduced the concept of \(f\)-divergence, which is a measure of deviation of two probability distributions, like e.g. Pearson’s \(\chi^2\)-deviation, which is given in terms of a convex function \(f\) defined on \([0,\infty)\). I. Vajda [Period. Math. Hung. 2, 223–234 (1972; Zbl 0248.62001); Inform. Theory, Statist. Decision Funct., Random Processes; Transact. 6th Prague Conf. 1971, 873–886 (1973; Zbl 0297.62003)] extended Pearson’s \(\chi^2\)-deviation to the family of the so-called \(\chi^\alpha\)-divergences, \(\alpha\in[1,\infty)\).
F. Österreicher and I. Vajda [Ann. Inst. Stat. Math. 55, No. 3, 639–653 (2003; Zbl 1052.62002)] introduced a family of \(f\)-divergences closely linked to the class \[ h_\alpha(t)=\begin{cases} \frac{1}{1-\alpha}[1-(t^{1/\alpha}+(1-t)^{1/\alpha})^\alpha] & \text{if }\alpha=(0,\infty)\setminus\{1\}\\ -[t\ln t+(1-t)\ln(1-t)] & \text{if }\alpha=1\\ \min(t,1-t) & \text{if }\alpha =0\end{cases} \] of Arimoto’s entropies S. Arimoto [Inf. Control 19, 181–194 (1971; Zbl 0222.94022)]. I. Vajda [Kybernetika 45, No. 6, 885–900 (2009; Zbl 1186.94421)] extended the corresponding family \(I_{\varphi_\alpha}\) of \(f\)-divergences to all \(\alpha\in\mathbb{R}\).
In Section 2 we consequently extend Arimoto’s class of entropies to all \(\alpha\in\mathbb{R}\). Theorem 3 in Section 4 relates the family \(I_{\varphi_\alpha}\) of \(f\)-divergences to Fisher’s information in a limiting way for all \(\alpha\in\mathbb{R}\) except for those from a certain neighbourhood of \(\alpha=0\). Its proof relies on an inequality of the form \[ \left|I_{\varphi_\alpha}(Q,P)-\frac{\varphi^{\prime\prime}_\alpha (1)}{2}\cdot\chi^2(Q,P)\right|\leq c_\alpha\cdot\chi^3(Q,P), \] which may be interesting also in its own right. This family of inequalities and its basic analytic counterpart are stated in Section 3. The corresponding proof is postponed to the Appendix.
Section 2 was stimulated by Vajda’s paper of 2009 and Section 4 considerably by his paper of 1973.


62B10 Statistical aspects of information-theoretic topics
94A17 Measures of information, entropy
Full Text: Link