×

Network inference combining mutual information rate and statistical tests. (English) Zbl 1501.94018

Summary: In this paper, we present a method that combines information-theoretical and statistical approaches to infer connectivity in complex networks using time-series data. The method is based on estimations of the Mutual Information Rate for pairs of time-series and on statistical significance tests for connectivity acceptance using the false discovery rate method for multiple hypothesis testing. We provide the mathematical background on Mutual Information Rate, discuss the statistical significance tests and the false discovery rate. Further on, we present results for correlated normal-variates data, coupled circle and coupled logistic maps, coupled Lorenz systems and coupled stochastic Kuramoto phase oscillators. Following up, we study the effect of noise on the presented methodology in networks of coupled stochastic Kuramoto phase oscillators and of coupling heterogeneity degree on networks of coupled circle maps. We show that the method can infer the correct number and pairs of connected nodes, by means of receiver operating characteristic curves. In the more realistic case of stochastic data, we demonstrate its ability to infer the structure of the initial connectivity matrices. The method is also shown to recover the initial connectivity matrices for dynamics on the nodes of Erdős-Rényi and small-world networks with varying coupling heterogeneity in their connections. The highlight of the proposed methodology is its ability to infer the underlying network connectivity based solely on the recorded datasets.

MSC:

94A20 Sampling theory in information and communication theory
94A15 Information theory (general)
94A17 Measures of information, entropy
62M20 Inference from stochastic processes and prediction

Software:

IOTA; MVGC
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Thurner, S.; Hanel, R.; Klimek, P., Introduction to the theory of complex systems (2018), Oxford University Press · Zbl 1400.37001
[2] Sayama, H., Introduction to the modeling and analysis of complex systems (2015), SUNY Binghamton
[3] Barrat, A.; Barthelemy, M.; Vespignani, A., Dynamical processes on complex networks (2008), Cambridge University Press · Zbl 1198.90005
[4] Niso, G.; Bruna, R.; Pereda, E.; Gutiérrez, R.; Bajo, R.; Maestú, F.; del Pozo, F., HERMES: Towards an integrated toolbox to characterize functional and effective brain connectivity, Rev Modern Phys, 11, 405-434 (2013)
[5] Omony, J., Biological network inference: A review of methods and assessment of tools and techniques, Annu Res Rev Biol, 4, 4, 577-601 (2014)
[6] Sakkalis, V., Review of advanced techniques for the estimation of brain connectivity measured with EEG/MEG, Comput Biol Med, 41, 12, 1110-1117 (2011)
[7] Olejarczyk, E.; Marzetti, L.; Pizzella, V.; Zappasodi, F., Comparison of connectivity analyses for resting state EEG data, J Neural Eng, 14, 3, Article 036017 pp. (2017)
[8] Jahanpour, E.; Chen, X., Improving accuracy of complex network modeling using maximum likelihood estimation and expectation-maximization, Interdiscip J Discontin, Nonlinearity, Complex, 3, 169-221 (2014)
[9] Rubido, N.; Martí, A. C.; Bianco-Martínez, E.; Grebogi, C.; Baptista, M. S.; Masoller, C., Exact detection of direct links in networks of interacting dynamical units, New J Phys, 16, Article 093010 pp. (2014)
[10] Tirabassi, G.; Sevilla-Escoboza, R.; Buldú, J. M.; Masoller, C., Inferring the connectivity of coupled oscillators from time-series statistical similarity analysis, Sci Rep, 5, 10829 (2015)
[11] Butte, A.; Kohane, I. S., Mutual information relevance networks: Functional genomic clustering using pairwise entropy measurements, Pac Symp Biocomput, 5, 415-426 (2000)
[12] Zou, Y.; Romano, M. C.; Thiel, M.; Marwan, N.; Kurths, J., Inferring indirect coupling by means of recurrences, Chaos, 21, 4, 1099-1111 (2011) · Zbl 1247.37095
[13] Nawrath, J.; Romano, M. C.; Thiel, M.; Kiss, I. Z.; Wickramasinghe, M.; Timmer, J.; Kurths, J.; Schelter, B., Distinguishing direct from indirect interactions in oscillatory networks with multiple time scales, Phys Rev Lett, 104, Article 038701 pp. (2010)
[14] Hempel, S.; Koseska, A.; Kurths, J.; Nikoloski, Z., Inner composition alignment for inferring directed networks from short time series, Phys Rev Lett, 107, Article 054101 pp. (2011)
[15] Ta, X. H.; Yoon, C. N.; Holm, L.; Han, S. K., Inferring the physical connectivity of complex networks from their functional dynamics, BMC Sys Biol, 4, 70, 1-12 (2010)
[16] Timme, M., Revealing network connectivity from response dynamics, Phys Rev Lett, 98, Article 224101 pp. (2007)
[17] Shandilya, S. G.; Timme, M., Inferring network topology from complex dynamics, New J Phys, 13, Article 013004 pp. (2011) · Zbl 1448.37045
[18] Timme, M.; Casadiego, J., Revealing networks from dynamics: An introduction, J Phys A, 47, Article 343001 pp. (2014) · Zbl 1305.92036
[19] Granger, C. W.J., Investigating causal relations by econometric models and cross-spectral methods, Econometrica, 37, 424-438 (1969) · Zbl 1366.91115
[20] Guo, S.; Ladroue, C.; Feng, J., Granger causality: theory and applications, (Front. comput. sys. biol., vol. 15 (2010)) · Zbl 1417.92054
[21] Bressler, S. L.; Seth, A. K., Wiener-Granger causality: A well established methodology, NeuroImage, 58, 2, 323-329 (2010)
[22] Barnett, L.; Seth, A. K., The MVGC multivariate Granger causality toolbox: A new approach to Granger-causal inference, J Neurosci Methods, 223, 50-68 (2014)
[23] Leguia, M. G.; Martínez, C. G.B.; Malvestio, I.; Campo, A. T.; Rocamora, R.; Levnajić, Z.; Andrzejak, R. G., Inferring directed networks using a rank-based connectivity measure, Phys Rev E, 99, 012319, Article 012319 pp. (2019)
[24] Chicharro, D.; Andrzejak, R. G., Reliable detection of directional couplings using rank statistics, Phys Rev E, 80, Article 026217 pp. (2009)
[25] Schiff, S. J.; So, P.; Chang, T.; Burke, R. E.; Sauer, T., Detecting dynamical interdependence and generalized synchrony through mutual prediction in a neural ensemble, Phys Rev E, 54, 6708 (1996)
[26] Lobier, M.; Siebenhühner, F.; Palma, S.; Palva, J. M., Phase transfer entropy: A novel phase-based measure for directed connectivity in networks coupled by oscillatory interactions, NeuroImage, 85, 853-872 (2014)
[27] Alderisio, F.; Fiore, G.; di Bernardo, M., Reconstructing the structure of directed and weighted networks of nonlinear oscillators, Phys Rev E, 95, Article 042302 pp. (2017), URL https://link.aps.org/doi/10.1103/PhysRevE.95.042302
[28] Kralemann, B.; Pikovsky, A.; Rosenblum, M., Reconstructing effective phase connectivity of oscillator networks from observations, New J Phys, 16, 8, Article 085013 pp. (2014)
[29] Yang, C.-L.; Suh, C. S., A general framework for dynamic complex networks, J Vib Test Syst Dyn, 5, 1, 87 (2021)
[30] Steuer, R.; Kurths, J.; Daub, C. O.; Weise, J.; Selbig, J., The mutual information: Detecting and evaluating dependencies between variables, Bioinformatics, 18, S231-S240 (2002)
[31] Palus̆, M., From nonlinearity to causality: Statistical testing and inference of physical mechanisms underlying complex dynamics, Contemp Phys, 48, 6, 307-348 (2007)
[32] Palus̆, M.; Hartman, D.; Hlinka, J.; Vejmelka, M., Discerning connectivity from dynamics in climate networks, Nonlinear Proc Geophys, 18, 751-763 (2011)
[33] Schreiber, T., Measuring information transfer, Phys Rev Lett, 85, 2, 461-464 (2000)
[34] Quiroga, R. Q.; Arnhold, J.; Grassberger, P., Learning driver-response relationships from synchronization patterns, Phys Rev E, 61, 5142-5148 (2000)
[35] Smirnov, D. A.; Bezruchko, B. P., Estimation of interaction strength and direction from short and noisy time series, Phys Rev E, 68, Article 046209 pp. (2003)
[36] Nalatore, H.; Ding, M.; Rangarajan, G., Mitigating the effects of measurement noise on Granger causality, Phys Rev E, 75, Article 031123 pp. (2007)
[37] Bianco-Martínez, E.; Rubido, N.; Antonopoulos, C. G.; Baptista, M. S., Successful network inference from time-series data using Mutual Information Rate, Chaos, 26, 4, Article 043102 pp. (2016) · Zbl 1361.37066
[38] Goh, Y. K.; Hasim, H. M.; Antonopoulos, C. G., Inference of financial networks using the normalised mutual information rate, PLoS One, 13, 2, 1-21 (2018)
[39] Baptista, M. S.; Rubinger, R. M.; Viana, E. R.; Sartorelli, J. C.; Parlitz, U.; Grebogi, C., Mutual information rate and bounds for it, PLoS One, 7, 10, Article e46745 pp. (2012)
[40] Palus, M., Coarse-grained entropy rates for characterization of complex time series, Physica D, 93, 64-77 (1996) · Zbl 0887.92002
[41] Herzel, H.; Schmitt, A. O.; Ebeling, W., Finite sample effects in sequence analysis, Chaos Solitons Fractals, 4, 1, 97-113 (1994) · Zbl 0790.62012
[42] J., C., Erratum: Between order and chaos, Nat Phys, 9, 382 (2013)
[43] Paluš, M., Coarse-grained entropy rates for characterization of complex time series, Physica D, 93, 1-3, 64-77 (1996) · Zbl 0887.92002
[44] Marchiori, M. A.; Fariello, R.; de Aguiar, M. A.M., Energy transfer dynamics and thermalization of two oscillators interacting via chaos, Phys Rev E, 85, 4, Article 041119 pp. (2012)
[45] Mandal, D.; Quan, H. T.; Jarzynski, C., Maxwell’s refrigerator: An exactly solvable model, Phys Rev Lett, 111, 3, Article 030602 pp. (2013)
[46] Antonopoulos, C. G.; Bianco-Martínez, E.; Baptista, M. S., Production and transfer of energy and information in Hamiltonian systems, PLoS One, 9, 2, Article e89585 pp. (2014)
[47] Antonopoulos, C. G.; Srivastava, S.; Pinto, S. E.S.; Baptista, M. S., Do brain networks evolve by maximizing their information flow capacity?, PLOS Comput Biol, 11, 8, Article e1004372 pp. (2015)
[48] Shannon, C. E., A mathematical theory of communication, Bell Labs Tech J, 27, 3, 379-423 (1948) · Zbl 1154.94303
[49] Kullback, S., Information theory and statistics (1997), Dover Publications · Zbl 0897.62003
[50] Eckmann, J. P.; Ruelle, D., Ergodic theory of chaos and strange attractors, Rev Modern Phys, 57, 3, 617 (1985) · Zbl 0989.37516
[51] Baptista, M. S.; Carvalho, J. X.D.; Hussein, M. S., Finding quasi-optimal network topologies for information transmission in active networks, PLoS One, 3, 10, Article e3479 pp. (2008)
[52] Pinto, P. R.F.; Baptista, M. S.; Labouriau, I. S., Density of first , Poincaré returns, periodic orbits, and, Kolmogorov-Sinai entropy, Commun Nonlinear Sci Numer Simul, 16, 2, 863-875 (2011) · Zbl 1221.37032
[53] Moddemeijer, R., On estimation of entropy and mutual information of continuous distributions, Signal Proc, 16, 3, 233-248 (1989)
[54] Moon, Y. I.; Rajagopalan, B.; Lall, U., Estimation of mutual information using kernel density estimators, Phys Rev E, 52, 3, 2318 (1995)
[55] Kraskov, A.; Stögbauer, H.; Grassberger, P., Estimating mutual information, Phys Rev E, 69, 6, Article 066138 pp. (2004)
[56] Butte AJ, Kohane IS. Mutual information relevance networks: Functional genomic clustering using pairwise entropy measurements. In: Pacific symposium on biocomputing, vol. 5. World Scientific; 2000, p. 418-29.
[57] Dobrushin, R. L., General formulation of Shannon’s main theorem in information theory, Am Math Soc Transl, 33, 2, 323-438 (1963) · Zbl 0132.39802
[58] Gray, R. M.; Kieffer, J. C., Asymptotically mean stationary measures, Ann Probab, 8, 5, 962-973 (1980) · Zbl 0447.28014
[59] Johnson, D. B., Efficient algorithms for shortest paths in sparse networks, J ACM, 24, 1-3, 1-13 (1997) · Zbl 0343.68028
[60] Prichard, D.; Theiler, J., Generating surrogate data for time series with several simultaneously measured variables, Phys Rev Lett, 73, 7, 951-954 (1994)
[61] Gómez-Gardeñes, J.; Zamora-López, G.; Moreno, Y.; Arenas, A., From modular to centralized organization of synchronization in functional areas of the cat cerebral cortex, PLoS One, 5, Article e12313 pp. (2010)
[62] Benjamini, Y.; Yekutieli, D., The control of the false discovery rate in multiple testing under dependency, Ann Statist, 29, 1165-1188 (2001) · Zbl 1041.62061
[63] Junior, L. S.; Mullokandov, A.; Kenett, D. Y., Dependency relations among international stock market indices, J Risk Financ Manag, 8, 2, 227-265 (2015)
[64] Kaneko, K., Clustering, coding, switching, hierarchical ordering, and control in a network of chaotic elements, Physica D, 41, 2, 137-172 (1990) · Zbl 0709.58520
[65] Acebrón, J. A.; Bonilla, L. L.; Vicente, C. J.P.; Ritort, F.; Spigler, R., The Kuramoto model: A simple paradigm for synchronization phenomena, Rev Modern Phys, 77, 137 (2005)
[66] Kaneko, K., Overview of coupled map lattices, Chaos, 2, 3, 279-282 (1992) · Zbl 1055.37540
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.