zbMATH — the first resource for mathematics

Learning Gaussian graphical models with fractional marginal pseudo-likelihood. (English) Zbl 1407.62193
Summary: We propose a Bayesian approximate inference method for learning the dependence structure of a Gaussian graphical model. Using pseudo-likelihood, we derive an analytical expression to approximate the marginal likelihood for an arbitrary graph structure without invoking any assumptions about decomposability. The majority of the existing methods for learning Gaussian graphical models are either restricted to decomposable graphs or require specification of a tuning parameter that may have a substantial impact on learned structures. By combining a simple sparsity inducing prior for the graph structures with a default reference prior for the model parameters, we obtain a fast and easily applicable scoring function that works well for even high-dimensional data. We demonstrate the favourable performance of our approach by large-scale comparisons against the leading methods for learning non-decomposable Gaussian graphical models. A theoretical justification for our method is provided by showing that it yields a consistent estimator of the graph structure.

62H12 Estimation in multivariate analysis
62F15 Bayesian inference
68T05 Learning and adaptive systems in artificial intelligence
PDF BibTeX Cite
Full Text: DOI
[1] Dempster, A. P., Covariance selection, Biometrics, 28, 1, 157-175, (1972)
[2] Whittaker, J., Graphical models in applied multivariate statistics, (1990), John Wiley & Sons · Zbl 0732.62056
[3] Lauritzen, S., Graphical models, (1996), Clarendon Press · Zbl 0907.62001
[4] Jones, B.; Carvalho, C.; Dobra, A.; Hans, C.; Carter, C.; West, M., Experiments in stochastic computation for high-dimensional graphical models, Stat. Sci., 20, 4, 388-400, (2005) · Zbl 1130.62408
[5] Scott, J. G.; Carvalho, C. M., Feature-inclusion stochastic search for Gaussian graphical models, J. Comput. Graph. Stat., 17, 4, 790-808, (2008)
[6] Carvalho, C. M.; Scott, J. G., Objective Bayesian model selection in Gaussian graphical models, Biometrika, 96, 3, 497-512, (2009) · Zbl 1170.62020
[7] Fitch, A. M.; Jones, M. B.; Massam, H., The performance of covariance selection methods that consider decomposable models only, Bayesian Anal., 9, 3, 659-684, (2014) · Zbl 1327.62389
[8] Wong, F.; Carter, C. K.; Kohn, R., Efficient estimation of covariance selection models, Biometrika, 90, 4, 809-830, (2003) · Zbl 1436.62346
[9] Atay-Kayis, A.; Massam, H., A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models, Biometrika, 92, 2, 317-335, (2005) · Zbl 1094.62028
[10] Moghaddam, B.; Khan, E.; Murphy, K. P.; Marlin, B. M., Accelerating Bayesian structural inference for non-decomposable Gaussian graphical models, (Bengio, Y.; Schuurmans, D.; Lafferty, J.; Williams, C.; Culotta, A., Adv. Neural Inf. Process. Syst., vol. 22, (2009), Curran Associates, Inc.), 1285-1293
[11] Dobra, A.; Lenkoski, A.; Rodriguez, A., Bayesian inference for general Gaussian graphical models with application to multivariate lattice data, J. Am. Stat. Assoc., 106, 496, 1418-1433, (2011) · Zbl 1234.62018
[12] Mohammadi, A.; Wit, E. C., Bayesian structure learning in sparse Gaussian graphical models, Bayesian Anal., 10, 1, 109-138, (2015) · Zbl 1335.62056
[13] Stingo, F.; Marchetti, G. M., Efficient local updates for undirected graphical models, Stat. Comput., 25, 1, 159-171, (2015) · Zbl 1331.62166
[14] Wang, H., Scaling it up: stochastic search structure learning in graphical models, Bayesian Anal., 10, 2, 351-377, (2015) · Zbl 1335.62068
[15] Friedman, J.; Hastie, T.; Tibshirani, R., Sparse inverse covariance estimation with the graphical lasso, Biostatistics, 9, 3, 432-441, (2008) · Zbl 1143.62076
[16] Witten, D. M.; Friedman, J. H.; Simon, N., New insights and faster computations for the graphical lasso, J. Comput. Graph. Stat., 20, 4, 892-900, (2011)
[17] Meinshausen, N.; Bühlmann, P., High-dimensional graphs and variable selection with the lasso, Ann. Stat., 34, 3, 1436-1462, (2006) · Zbl 1113.62082
[18] Peng, J.; Wang, P.; Zhou, N.; Zhu, J., Partial correlation estimation by joint sparse regression models, J. Am. Stat. Assoc., 104, 486, 735-746, (2009) · Zbl 1388.62046
[19] Dobra, A.; Hans, C.; Jones, B.; Nevins, J. R.; Yao, G.; West, M., Sparse graphical models for exploring gene expression data, J. Multivar. Anal., 90, 1, 196-212, (2004), special Issue on Multivariate Methods in Genomic Data Analysis · Zbl 1047.62104
[20] Heckerman, D.; Chickering, D. M.; Meek, C.; Rounthwaite, R.; Kadie, C., Dependency networks for inference, collaborative filtering, and data visualization, J. Mach. Learn. Res., 1, 49-75, (2001) · Zbl 1008.68132
[21] Pensar, J.; Nyman, H.; Niiranen, J.; Corander, J., Marginal pseudo-likelihood learning of discrete Markov network structures, Bayesian Anal. Adv. Publ., (2016)
[22] Consonni, G.; La Rocca, L., Objective Bayes factors for Gaussian directed acyclic graphical models, Scand. J. Stat., 39, 4, 743-756, (2012) · Zbl 1253.62015
[23] O’Hagan, A., Fractional Bayes factors for model comparison, J. R. Stat. Soc. B, 57, 1, 99-138, (1995) · Zbl 0813.62026
[24] Geiger, D.; Heckerman, D., Parameter priors for directed acyclic graphical models and the characterization of several probability distributions, Ann. Stat., 30, 5, 1412-1440, (2002) · Zbl 1016.62064
[25] Besag, J. E., Nearest-neighbour systems and the auto-logistic model for binary data, J. R. Stat. Soc. B, 34, 1, 75-83, (1972) · Zbl 0268.60088
[26] Koller, D.; Friedman, N., Probabilistic graphical models: principles and techniques, (2009), MIT Press
[27] Haughton, D. M.A., On the choice of a model to fit data from an exponential family, Ann. Stat., 16, 1, 342-355, (1988) · Zbl 0657.62037
[28] Wei, C. Z., On predictive least squares principles, Ann. Stat., 20, 1, 1-42, (1992) · Zbl 0801.62083
[29] Chickering, D. M., Optimal structure identification with greedy search, J. Mach. Learn. Res., 3, 507-554, (2003) · Zbl 1084.68519
[30] Tsamardinos, I.; Aliferis, C. F.; Statnikov, A., Algorithms for large scale Markov blanket discovery, (Russell, I.; Haller, S., The 16th International FLAIRS Conference, (2003), AAAI Press), 376-380
[31] Peña, J. M.; Nilsson, R.; Björkegren, J.; Tegnér, J., Towards scalable and data efficient learning of Markov boundaries, Int. J. Approx. Reason., 45, 2, 211-232, (2007) · Zbl 1122.68136
[32] Tsamardinos, I.; Brown, L. E.; Aliferis, C. F., The MAX-MIN Hill-climbing Bayesian network structure learning algorithm, Mach. Learn., 65, 1, 31-78, (2006)
[33] Gelman, A.; Carlin, J. B.; Stern, H. S.; Dunson, D. B.; Vehtari, A.; Rubin, D. B., Bayesian data analysis, (2014), Chapman and Hall/CRC · Zbl 1279.62004
[34] Foygel, R.; Drton, M., Extended Bayesian information criteria for Gaussian graphical models, (Lafferty, J. D.; Williams, C. K.I.; Shawe-Taylor, J.; Zemel, R. S.; Culotta, A., Adv. Neural Inf. Process. Syst., vol. 23, (2010), Curran Associates, Inc.), 604-612
[35] Friedman, J.; Hastie, T.; Tibshirani, R., R-package glasso: graphical lasso-estimation of Gaussian graphical models, (2014), (accessed 27 September 2016)
[36] Peng, J.; Wang, P.; Zhou, N.; Zhu, J., R-package space: sparse partial correlation estimation, (2010), (accessed 27 September 2016)
[37] Achard, S., R-package brainwaver: basic wavelet analysis of multivariate time series with a visualisation and parametrisation using graph theory, (2012), (accessed 27 September 2016)
[38] Hiissa, J.; Elo, L. L.; Huhtinen, K.; Perheentupa, A.; Poutanen, M.; Aittokallio, T., Resampling reveals sample-level differential expression in clinical genome-wide studies, Omics. J. Integr. Biol., 13, 5, 381-396, (2009)
[39] Altomare, D.; Consonni, G.; La Rocca, L., R-package fbfsearch: algorithm for searching the space of Gaussian directed acyclic graphical models through moment fractional Bayes factors, (2013), (accessed 27 September 2016)
[40] Sachs, K.; Perez, O.; Pe’er, D.; Lauffenburger, D. A.; Nolan, G. P., Causal protein-signaling networks derived from multiparameter single-cell data, Science, 308, 5721, 523-529, (2005)
[41] Sun, H.; Li, H., Robust Gaussian graphical modeling via l1 penalization, Biometrics, 68, 4, 1197-1206, (2012) · Zbl 1259.62102
[42] Press, J. S., Applied multivariate analysis: using Bayesian and frequentist method of inference, (1982), Robert E. Krieger Publishing Company
[43] Baldi, P.; Brunak, S.; Chauvin, Y.; Andersen, C. A.F.; Nielsen, H., Assessing the accuracy of prediction algorithms for classification: an overview, Bioinformatics, 16, 5, 412-424, (2000)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.