×
Ay, Nihat; Jost, Jürgen; Lê, Hông Vân; Schwachhöfer, Lorenz

Information geometry and sufficient statistics. (English) Zbl 1321.62008

Probab. Theory Relat. Fields 162, No. 1-2, 327-364 (2015).
Summary: Information geometry provides a geometric approach to families of statistical models. The key geometric structures are the Fisher quadratic form and the Amari-Chentsov tensor. In statistics, the notion of sufficient statistic expresses the criterion for passing from one model to another without loss of information. This leads to the question how the geometric structures behave under such sufficient statistics. While this is well studied in the finite sample size case, in the infinite case, we encounter technical problems concerning the appropriate topologies. Here, we introduce notions of parametrized measure models and tensor fields on them that exhibit the right behavior under statistical transformations. Within this framework, we can then handle the topological issues and show that the Fisher metric and the Amari-Chentsov tensor on statistical models in the class of symmetric 2-tensor fields and 3-tensor fields can be uniquely (up to a constant) characterized by their invariance under sufficient statistics, thereby achieving a full generalization of the original result of Chentsov to infinite sample sizes. More generally, we decompose Markov morphisms between statistical models in terms of statistics. In particular, a monotonicity result for the Fisher information naturally follows.

Cited in 1 Review
Cited in 38 Documents

MSC:

62B05 Sufficient statistics and fields
53B20 Local Riemannian geometry
62B10 Statistical aspects of information-theoretic topics

Keywords:

Fisher quadratic form; Amari-Chentsov tensor; sufficient statistic; Chentsov theorem
PDFBibTeX XMLCite
\textit{N. Ay} et al., Probab. Theory Relat. Fields 162, No. 1--2, 327--364 (2015; Zbl 1321.62008)
Full Text: DOI arXiv

References:

[1] Amari, S.: Theory of information spaces a geometrical foundation of statistics. POST RAAG Report 106 (1980) · Zbl 0433.62004
[2] Amari, S.: Differential geometry of curved exponential families curvature and information loss. Ann. Stat. 10, 357-385 (1982) · Zbl 0507.62026 · doi:10.1214/aos/1176345779
[3] Amari, S.: Differential geometrical theory of statistics. In: Differential Geometry in Statistical Inference. Lecture Note-Monograph Series, vol. 10. Institute of Mathematical Statistics, California (1987) · Zbl 0693.62034
[4] Amari, S., Nagaoka, H.: Methods of Information Geometry. Translations of Mathematical Monographs, vol. 191, American Mathematical Society, Providence (2000) · Zbl 0960.62005
[5] Ay, N.: An Information-geometric approach to a theory of pragmatic structuring. Ann. Probab. 30(1), 416-436 (2002) · Zbl 1010.62007 · doi:10.1214/aop/1020107773
[6] Ay, N., Olbrich, E., Bertschinger, N., Jost, J.: A geometric approach to complexity. Chaos 21, 37-103 (2011) · Zbl 1317.37039 · doi:10.1063/1.3638446
[7] Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L.: Information geometry (book in preparation) · Zbl 1383.53002
[8] Blackwell, D.: Equivalent comparisons of experiments. Ann. Math. Stat. 24, 265-272 (1953) · Zbl 0050.36004 · doi:10.1214/aoms/1177729032
[9] Borovkov, A.A.: Mathematical Statistics. Gordon and Breach Science Publishers, Amsterdam (1998) · Zbl 0913.62002
[10] Campbell, L.L.: An extended Chentsov characterization of a Riemannian metric. Proc. AMS 98, 135-141 (1986) · Zbl 0608.62013
[11] Cena, A., Pistone, G.: Exponential statistical model. AISM 59, 27-56 (2007) · Zbl 1108.62003 · doi:10.1007/s10463-006-0096-y
[12] Cramer, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946) · Zbl 0063.01014
[13] Chentsov, N.: Category of mathematical statistics. Dokl. Acad. Nauk USSR 164, 511-514 (1965) · Zbl 0142.15102
[14] Chentsov, N.: Algebraic foundation of mathematical statistics. Math. Operationsforsch. Stat. Ser. Stat. 9, 267-276 (1978) · Zbl 0387.62002
[15] Chentsov, N.: Statistical Decision Rules and Optimal Inference. Translation of Mathematical Monograph, vol. 53. AMS, Providence (1982) · Zbl 0484.62008
[16] Efron, B.: Defining the curvature of a statistical problem (with applications to second order efficiency), with a discussion by C. R. Rao, Don A. Pierce, D. R. Cox, D. V. Lindley, Lucien LeCam, J. K. Ghosh, J. Pfanzagl, Niels Keiding, A. P. Dawid, Jim Reeds and with a reply by the author. Ann. Stat. 3, 1189-1242 (1975) · Zbl 0321.62013 · doi:10.1214/aos/1176343282
[17] Fisher, R.A.: On the mathematical foundations of theoretical statistics. Philos. Trans. R. Soc. Lond. Ser. A 222, 309-368 (1922) · JFM 48.1280.02
[18] Gibilisco, P., Pistone, G.: Connections on non-parametric statistical models by Orlicz space geometry. Infin. Dimens. Anal. Quantum Probab. Relat. Top. 1(2), 325-347 (1998) · Zbl 0921.62004 · doi:10.1142/S021902579800017X
[19] Hamilton, R.: The inverse function theorem of Nash Moser. Bull. AMS. 7, 65-222 (1982) · Zbl 0499.58003 · doi:10.1090/S0273-0979-1982-15004-2
[20] Jeffreys, H.: An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. Ser. A 186, 453-461 (1946) · Zbl 0063.03050 · doi:10.1098/rspa.1946.0056
[21] Kallenberg, O.: Foundations of Modern Probability, 2nd edn. Springer, New York (2001) · Zbl 0996.60001
[22] Krasnoselskii, M.A., Rutickii, Ya.B.: Convex functions and Orlicz spaces. Fizmatgiz, Moskava (1958) [In Russian; English translation: P. Noordfoff Ltd., Groningen (1961)] · Zbl 0050.36004
[23] Kullback, S.: Information Theory and Statistics. Dover Publications, New York (1968) · Zbl 0897.62003
[24] Kass, R., Vos, P.W.: Geometrical foundations of asymptotic inference. Wiley, New York (1997) · Zbl 0880.62005 · doi:10.1002/9781118165980
[25] Lauritzen, S.: Statistical manifolds. In: Differential Geometry in Statistical Inference. Lecture Note-Monograph Series, vol. 10. Institute of Mathematical Statistics, California (1987) · Zbl 0694.62001
[26] Lê, H.V.: Statistical manifolds are statistical models. J. Geom. 84, 83-93 (2005) · Zbl 1101.53037 · doi:10.1007/s00022-005-0030-0
[27] Lê, H.V.: Monotone invariants and embedding of statistical models. In: Advances in Deterministic and Stochastic Analysis. World Scientific, pp. 231-254 (2007). arXiv:math/0506163 · Zbl 0507.62026
[28] Lê, H.V.: The uniqueness of the Fisher metric as information metric. arXiv:1306.1465 · Zbl 1393.62004
[29] Lê, H.V., Somberg, P., Vanžura, J.: Smooth structures on pseudomanifolds with isolated conical singularities. Acta Math. Vietnam. 38, 33-54 (2013). ArXiv:1006.5707 · Zbl 1273.53068
[30] Morse, N., Sacksteder, R.: Statistical isomorphism. Ann. Math. Stat. 37, 203-214 (1966) · Zbl 0158.37105 · doi:10.1214/aoms/1177699610
[31] Morozova, E., Chentsov, N.: Markov invariant geometry on manifolds of states. In: Itogi Nauki i Techniki, Current Problems of Mathematics, Newest Achievements, vol. 6. Moscow, pp. 69-102 (1990) · Zbl 0727.60006
[32] Morozova, E., Chentsov, N.: Natural geometry on families of probability laws. In: Itogi Nauki i Techniki, Current Problems of Mathematics. Fundamental Directions, vol. 83, Moscow, pp. 133-265 (1991) · Zbl 0753.62011
[33] Murray, M., Rice, J.: Differential Geometry and Statistics. Chapman and Hall, London (1993) · Zbl 0804.53001 · doi:10.1007/978-1-4899-3306-5
[34] Olbrich, E., Kahle, T., Bertschinger, N., Ay, N., Jost, J.: Quantifying structure in networks. Eur. Phys. J. B Condens. Matter Complex Syst. 77, 239-247 (2010)
[35] Pistone, G., Sempi, C.: An infinite-dimensional structure on the space of all the probability measures equivalent to a given one. Ann. Stat. 5, 1543-1561 (1995) · Zbl 0848.62003 · doi:10.1214/aos/1176324311
[36] Rao, C.R.: Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 37, 81-89 (1945) · Zbl 0063.06420
[37] Shahshahani, S.: A new mathematical framework for the study of linkage and selection. Memoirs of the American Mathematical Society, vol. 17, no. 211. American Mathematical Society (1979) · Zbl 0473.92008
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.