Operator-valued kernel-based vector autoregressive models for network inference. (English) Zbl 1359.62377

Summary: Reverse-engineering of high-dimensional dynamical systems from time-course data still remains a challenging and important problem in knowledge discovery. For this learning task, a number of approaches primarily based on sparse linear models or Granger causality concepts have been proposed in the literature. However, when a system exhibits nonlinear dynamics, there does not exist a systematic approach that takes into account the nature of the underlying system. In this work, we introduce a novel family of vector autoregressive models based on different operator-valued kernels to identify the dynamical system and retrieve the target network that characterizes the interactions of its components. Assuming a sparse underlying structure, a key challenge, also present in the linear case, is to control the model’s sparsity. This is achieved through the joint learning of the structure of the kernel and the basis vectors. To solve this learning task, we propose an alternating optimization algorithm based on proximal gradient procedures that learns both the structure of the kernel and the basis vectors. Results on the DREAM3 competition gene regulatory benchmark networks of sizes 10 and 100 show the new model outperforms existing methods. Another application of the model on climate data identifies interesting and interpretable interactions between natural and human activity factors, thus confirming the ability of the learning scheme to retrieve dependencies between state-variables.


62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62P10 Applications of statistics to biology and medical sciences; meta analysis
Full Text: DOI


[1] Aijo, T; Lahdesmaki, H, Learning gene regulatory networks from gene expression measurements using non-parametric molecular kinetics, Bioinformatics, 25, 2937-2944, (2009)
[2] Alvarez, M. A., Rosasco, L., & Lawrence, D. N. (2011). Kernels for vector-valued functions: A review. Technical report, MIT_CSAIL-TR-2011-033. · Zbl 1301.68212
[3] Auliac, C; Frouin, V; Gidrol, X, Evolutionary approaches for the reverse-engineering of gene regulatory networks: A study on a biologically realistic dataset, BMC Bioinformatics, 9, 91, (2008)
[4] Baldassarre, L., Rosasco, L., Barla, A., & Verri, A. (2010). Vector field learning via spectral filtering. In J. Balczar, F. Bonchi, A. Gionis, & M. Sebag (Eds.), Machine learning and knowledge discovery in databases. Lecture notes in computer science (Vol. 6321, pp. 56-71). Berlin/Heidelberg: Springer. · Zbl 1243.68233
[5] Beck, A; Teboulle, M; Palomar, D (ed.); Eldar, Y (ed.), Gradient-based algorithms with applications to signal recovery problems, 42-88, (2010), Cambridge · Zbl 1211.90290
[6] Bolstad, A; Veen, B; Nowak, R, Causal network inference via group sparsity regularization, IEEE Trans Signal Process, 59, 2628-2641, (2011) · Zbl 1392.94012
[7] Brouard, C., d’Alché Buc, F., & Szafranski, M. (2011). Semi-supervised penalized output kernel regression for link prediction. In ICML-2011 (pp. 593-600). · Zbl 1141.62030
[8] Bühlmann, P., & van de Geer, S. (2011). Statistics for high-dimensional data: Methods, theory and applications. Berlin: Springer. · Zbl 1273.62015
[9] Caponnetto, A., Micchelli, C. A., Pontil, M., & Ying, Y. (2008). Universal multitask kernels. The Journal of Machine Learning Research, 9, 1615-1646. · Zbl 1225.68155
[10] Chatterjee, S., Steinhaeuser, K., Banerjee, A., Chatterjee, S., & Ganguly, A. R. (2012). Sparse group lasso: Consistency and climate applications. In SDM (pp. 47-58). SIAM/Omnipress
[11] Chou, I; Voit, EO, Recent developments in parameter estimation and structure identification of biochemical and genomic systems, Mathematical Biosciences, 219, 57-83, (2009) · Zbl 1168.92019
[12] Combettes, P. L., & Pesquet, J. C. (2011). Proximal splitting methods in signal processing. In Fixed-point algorithms for inverse problems in science and engineering. Springer Optimization and Its Applications, Vol. 49, pp. 185-212. · Zbl 1242.90160
[13] Dinuzzo, F., & Fukumizu, K. (2011). Learning low-rank output kernels. In Proceedings of the 3rd Asian conference on machine learning, JMLR: Workshop and conference proceedings, Vol. 20.
[14] Dondelinger, F; Lèbre, S; Husmeier, D, Non-homogeneous dynamic Bayesian networks with Bayesian regularization for inferring gene regulatory networks with gradually time-varying structure, Machine Learning Journal, 90, 191-230, (2013) · Zbl 1260.92027
[15] Friedman, N, Inferring cellular networks using probabilistic graphical models, Science, 303, 799-805, (2004)
[16] Gilchrist, S; Yankov, V; Zakrajšek, E, Credit market shocks and economic fluctuations: evidence from corporate bond and stock markets, Journal of Monetary Economics, 56, 471-493, (2009)
[17] Hartemink, A, Reverse engineering gene regulatory networks, Nat Biotechnol, 23, 554-555, (2005)
[18] Iba, H, Inference of differential equation models by genetic programming, Information Sciences, 178, 4453-4468, (2008) · Zbl 0925.82089
[19] Kadri, H., Rabaoui, A., Preux, P., Duflos, E., & Rakotomamonjy, A. (2011). Functional regularized least squares classication with operator-valued kernels. In ICML-2011 (pp 993-1000).
[20] Kolaczyk, E. D. (2009). Statistical analysis of network data: Methods and models: Series in Statistics. Berlin: Springer. · Zbl 1277.62021
[21] Kramer, MA; Eden, UT; Cash, SS; Kolaczyk, ED, Network inference with confidence from multivariate time series, Physical Review E, 79, 061,916+, (2009)
[22] Lawrence, N., Girolami, M., Rattray, M., & Sanguinetti, G. (Eds.) (2010). Learning and inference in computational systems biology. Cambridge: MIT Press. · Zbl 1196.92018
[23] Lèbre, S, Inferring dynamic genetic networks with low order independencies, Statistical Applications in Genetics and Molecular Biology, 8, 1-38, (2009) · Zbl 1276.62080
[24] Lim, N; Senbabaoglu, Y; Michailidis, G, OKVAR-boost: A novel boosting algorithm to infer nonlinear dynamics and interactions in gene regulatory networks, Bioinformatics, 29, 1416-1423, (2013)
[25] Liu, Y., Niculescu-Mizil, A., & Lozano, A. (2010). Learning temporal causal graphs for relational time-series analysis. In J. Fürnkranz, & T. Joachims (Eds.), ICML-2010.
[26] Maathuis, M; Colombo, D; Kalish, M; Bühlmann, P, Predicting causal effects in large-scale systems from observational data, Nature Methods, 7, 247-248, (2010)
[27] Margolin, I; Nemenman, Aand, Aracne: an algorithm for the reconstruction of gene regulatory networks in a Mammalian cellular context, BMC Bioinformatics, 7, s7, (2006)
[28] Mazur, J; Ritter, D; Reinelt, G; Kaderali, L, Reconstructing nonlinear dynamic models of gene regulation using stochastic sampling, BMC Bioinformatics, 10, 448, (2009)
[29] Meinshausen, N; Bühlmann, P, High dimensional graphs and variable selection with the lasso, Annals of Statistics, 34, 1436-1462, (2006) · Zbl 1113.62082
[30] Micchelli, CA; Pontil, MA, On learning vector-valued functions, Neural Computation, 17, 177-204, (2005) · Zbl 1092.93045
[31] Michailidis, G, Statistical challenges in biological networks, Journal of Computational and Graphical Statistics, 21, 840-855, (2012)
[32] Michailidis, G; d’Alché Buc, F, Autoregressive models for gene regulatory network inference: sparsity, stability and causality issues, Mathematical Biosciences, 246, 326-334, (2013) · Zbl 1308.92032
[33] Morton, R., & Williams, K. C. (2010). Experimental political science and the study of causality. Cambridge: Cambridge University Press.
[34] Murphy, K. P. (1998). Dynamic bayesian networks: Representation, inference and learning. PhD thesis, Computer Science, University of Berkeley, CA, USA. · Zbl 1092.93045
[35] Parry, M., Canziani, O., Palutikof, J., van der Linden, P., Hanson, C., et al. (2007). Climate change 2007: Impacts, adaptation and vulnerability. Intergovernmental Panel on Climate Change.
[36] Perrin, B. E., Ralaivola, L., & Mazurie, A., Bottani, S., Mallet, J., d’Alché-Buc, F. (2003). Gene networks inference using dynamic bayesian networks. Bioinformatics, 19(S2), 38-48.
[37] Prill, R; Marbach, D; Saez-Rodriguez, J; Sorger, P; Alexopoulos, L; Xue, X; Clarke, N; Altan-Bonnet, G; Stolovitzky, G, Towards a rigorous assessment of systems biology models: the DREAM3 challenges, PLoS ONE, 5, e9202, (2010)
[38] Raguet, H., Fadili, & J., Peyré, G. (2011). Generalized forward-backward splitting. arXiv preprint arXiv:1108.4404.
[39] Richard, E; Savalle, PA; Vayatis, N; Langford, J (ed.); Pineau, J (ed.), Estimation of simultaneously sparse and low rank matrices, 1351-1358, (2012), New York, NY, USA
[40] Schaffter, T; Marbach, D; Floreano, D, Genenetweaver: in silico benchmark generation and performance profiling of network inference methods, Bioinformatics, 27, 2263-2270, (2011)
[41] Senkene, E., & Tempel’man, A. (1973). Hilbert spaces of operator-valued functions. Lithuanian Mathematical Journal, 13(4), 665-670.
[42] Shojaie, A; Michailidis, G, Discovering graphical Granger causality using a truncating lasso penalty, Bioinformatics, 26, i517-i523, (2010)
[43] Yuan, M; Lin, Y, Model selection and estimation in regression with grouped variables, Journal of the Royal Statistical Society: Series B, 68, 49-67, (2006) · Zbl 1141.62030
[44] Zou, C; Feng, J, Granger causality vs. dynamic Bayesian network inference: A comparative study, BMC Bioinformatics, 10, 122, (2009)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.