×

zbMATH — the first resource for mathematics

The relation between Granger causality and directed information theory: a review. (English) Zbl 06346067
Summary: This report reviews the conceptual and theoretical links between Granger causality and directed information theory. We begin with a short historical tour of Granger causality, concentrating on its closeness to information theory. The definitions of Granger causality based on prediction are recalled, and the importance of the observation set is discussed. We present the definitions based on conditional independence. The notion of instantaneous coupling is included in the definitions. The concept of Granger causality graphs is discussed. We present directed information theory from the perspective of studies of causal influences between stochastic processes. Causal conditioning appears to be the cornerstone for the relation between information theory and Granger causality. In the bivariate case, the fundamental measure is the directed information, which decomposes as the sum of the transfer entropies and a term quantifying instantaneous coupling. We show the decomposition of the mutual information into the sums of the transfer entropies and the instantaneous coupling measure, a relation known for the linear Gaussian case. We study the multivariate case, showing that the useful decomposition is blurred by instantaneous coupling. The links are further developed by studying how measures based on directed information theory naturally emerge from Granger causality inference frameworks as hypothesis testing.

MSC:
62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
94A17 Measures of information, entropy
PDF BibTeX XML Cite
Full Text: DOI arXiv
References:
[2] DOI: 10.1155/2007/13853 · Zbl 05735173 · doi:10.1155/2007/13853
[3] DOI: 10.2307/1912791 · Zbl 1366.91115 · doi:10.2307/1912791
[4] Sims, Money, income and causality, Am. Econ. Rev. 62 pp 540– (1972)
[5] Sporns, The Networks of the Brain (2010) · Zbl 1093.92028
[6] DOI: 10.1038/40332 · doi:10.1038/40332
[7] DOI: 10.1007/s007040170019 · doi:10.1007/s007040170019
[8] Wiener, The theory of prediction, Modern Mathematics for the Engineer pp 165– (1956)
[9] Granger, Times Series Anlaysis, Cointegration and Applications, Nobel Lecture (2003)
[10] DOI: 10.1109/TCOM.1973.1091610 · doi:10.1109/TCOM.1973.1091610
[13] DOI: 10.1103/PhysRevLett.85.461 · doi:10.1103/PhysRevLett.85.461
[14] DOI: 10.1016/j.physrep.2006.12.004 · doi:10.1016/j.physrep.2006.12.004
[15] DOI: 10.1007/s00422-006-0098-0 · Zbl 1161.62429 · doi:10.1007/s00422-006-0098-0
[16] DOI: 10.1080/00107510801959206 · doi:10.1080/00107510801959206
[17] Pearl, Causality: Models, Reasoning and Inference (2000)
[18] Lauritzen, Chapter 2: Causal inference from graphical models, Complex Stochastic Systems pp 63– (2001) · Zbl 1010.62004
[19] Lauritzen, Graphical Models (1996) · Zbl 0907.62001
[20] Whittaker, Graphical Models in Applied Multivariate Statistics (1989) · Zbl 1151.62053
[21] DOI: 10.1016/S0019-9958(63)90092-5 · Zbl 0123.37502 · doi:10.1016/S0019-9958(63)90092-5
[22] DOI: 10.1109/TAC.1975.1101008 · Zbl 0312.60018 · doi:10.1109/TAC.1975.1101008
[23] DOI: 10.2307/1913964 · Zbl 0366.62119 · doi:10.2307/1913964
[24] DOI: 10.2307/1912601 · Zbl 0483.60022 · doi:10.2307/1912601
[25] DOI: 10.2307/1912602 · Zbl 0483.60023 · doi:10.2307/1912602
[26] DOI: 10.1016/0304-4076(88)90045-0 · Zbl 04522350 · doi:10.1016/0304-4076(88)90045-0
[27] DOI: 10.1016/0165-1889(80)90069-X · doi:10.1016/0165-1889(80)90069-X
[28] DOI: 10.1007/s00440-011-0345-8 · Zbl 1316.60049 · doi:10.1007/s00440-011-0345-8
[29] Dahlaus, Causality and graphical models in time series analysis, Highly Structured Stochastic Systems (2003)
[30] DOI: 10.1007/s00422-006-0062-z · Zbl 1138.62048 · doi:10.1007/s00422-006-0062-z
[31] DOI: 10.1080/01621459.1982.10477803 · doi:10.1080/01621459.1982.10477803
[32] DOI: 10.1080/01621459.1984.10477110 · doi:10.1080/01621459.1984.10477110
[34] DOI: 10.1103/PhysRevLett.103.238701 · doi:10.1103/PhysRevLett.103.238701
[35] DOI: 10.1103/PhysRevE.81.041907 · doi:10.1103/PhysRevE.81.041907
[36] DOI: 10.1007/s10827-010-0231-x · doi:10.1007/s10827-010-0231-x
[37] DOI: 10.1007/s10827-010-0247-2 · doi:10.1007/s10827-010-0247-2
[38] Gouriéroux, Kullback causality measures, Ann. Econ. Stat. 369 pp 369– (1987)
[39] DOI: 10.1109/TIT.1987.1057325 · Zbl 0626.62090 · doi:10.1109/TIT.1987.1057325
[40] Kullback, Information Theory and Statistics (1968)
[41] DOI: 10.1103/PhysRevE.61.5142 · doi:10.1103/PhysRevE.61.5142
[43] DOI: 10.1016/S0167-2789(98)00258-9 · Zbl 1054.92502 · doi:10.1016/S0167-2789(98)00258-9
[44] Kantz, Nonlinear Time Series Analysis (2004)
[45] DOI: 10.1103/PhysRevE.63.046211 · doi:10.1103/PhysRevE.63.046211
[46] Frenzel, Partial mutual information for coupling analysis of multivariate time series, Phys. Rev. Lett. 99 pp 204101– (2007) · doi:10.1103/PhysRevLett.99.204101
[47] DOI: 10.1016/S0167-2789(02)00432-3 · Zbl 1009.94004 · doi:10.1016/S0167-2789(02)00432-3
[48] DOI: 10.1103/PhysRevE.75.056211 · doi:10.1103/PhysRevE.75.056211
[49] DOI: 10.1103/PhysRevE.77.026214 · doi:10.1103/PhysRevE.77.026214
[50] Eichler, A graphical approach for evaluating effective connectivity in neural systems, Phil. Trans. R. Soc. B 360 pp 953– (2005) · doi:10.1098/rstb.2005.1641
[51] Kaminski, Evaluating causal relations in neural systems: Granger causality, directed transfer functions and statistical assessment of significance, Biol. Cybern. 85 pp 145– (2001) · Zbl 1160.92314 · doi:10.1007/s004220000235
[52] DOI: 10.1371/journal.pcbi.0020144 · doi:10.1371/journal.pcbi.0020144
[53] DOI: 10.1175/JCLI3653.1 · doi:10.1175/JCLI3653.1
[54] Saito, Recent Advances in EEG and EMG Data Processing pp 133– (1981)
[55] Cover, Elements of Information Theory (2006)
[56] DOI: 10.1109/TIT.2008.917685 · Zbl 1328.94034 · doi:10.1109/TIT.2008.917685
[57] Tatikonda, Control Under Communication Constraints , PhD thesis (2000) · Zbl 1365.93271
[58] DOI: 10.1109/TIT.2008.2008147 · Zbl 1367.94187 · doi:10.1109/TIT.2008.2008147
[59] DOI: 10.1109/TIT.2007.896887 · Zbl 1325.94086 · doi:10.1109/TIT.2007.896887
[63] DOI: 10.1002/ecja.4400670602 · doi:10.1002/ecja.4400670602
[64] DOI: 10.1109/LSP.2008.2006332 · doi:10.1109/LSP.2008.2006332
[67] DOI: 10.1103/PhysRevE.77.056215 · doi:10.1103/PhysRevE.77.056215
[68] Lehmann, Theory of Point Estimation (1998) · Zbl 0916.62017
[70] DOI: 10.1103/PhysRevLett.108.258701 · doi:10.1103/PhysRevLett.108.258701
[71] DOI: 10.1109/TIT.2011.2136270 · Zbl 1365.94128 · doi:10.1109/TIT.2011.2136270
[73] DOI: 10.1109/TIT.1980.1056222 · Zbl 0452.94010 · doi:10.1109/TIT.1980.1056222
[74] Gray, Entropy and Information Theory (1990)
[75] Pinsker, Information and Information Stability of Random Variables (1964) · Zbl 0125.09202
[76] DOI: 10.1007/s10827-010-0262-3 · doi:10.1007/s10827-010-0262-3
[77] DOI: 10.1103/PhysRevLett.109.138105 · doi:10.1103/PhysRevLett.109.138105
[79] DOI: 10.1371/journal.pcbi.1001110 · doi:10.1371/journal.pcbi.1001110
[80] Meyn, Markov Chains and Stochastic Stability (2009)
[81] Schölkopf, Learning with Kernels (2002)
[82] Lehmann, Testing Statistical Hypotheses (2005) · Zbl 1076.62018
[83] Beirlant, Nonparametric entropy estimation: An overview, Int. J. Math. Stat. Sci. 6 pp 17– (1997) · Zbl 0882.62003
[84] DOI: 10.1080/104852504200026815 · Zbl 1061.62005 · doi:10.1080/104852504200026815
[85] DOI: 10.1103/PhysRevE.69.066138 · doi:10.1103/PhysRevE.69.066138
[86] Kozachenko, Sample estimate of the entropy of a random vector, Problems Inf. Trans. 23 pp 95– (1987) · Zbl 0633.62005
[87] DOI: 10.1162/089976603321780272 · Zbl 1052.62003 · doi:10.1162/089976603321780272
[88] DOI: 10.1214/07-AOS539 · Zbl 1205.94053 · doi:10.1214/07-AOS539
[89] DOI: 10.1109/TIT.2012.2195549 · Zbl 1365.62131 · doi:10.1109/TIT.2012.2195549
[90] DOI: 10.1109/TIT.2009.2016060 · Zbl 1367.94141 · doi:10.1109/TIT.2009.2016060
[91] DOI: 10.1016/j.sigpro.2012.09.003 · doi:10.1016/j.sigpro.2012.09.003
[92] DOI: 10.1016/j.physleta.2011.06.057 · Zbl 1250.82018 · doi:10.1016/j.physleta.2011.06.057
[94] DOI: 10.1109/TNNLS.2011.2178327 · doi:10.1109/TNNLS.2011.2178327
[95] DOI: 10.1016/j.jneumeth.2008.04.011 · doi:10.1016/j.jneumeth.2008.04.011
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.