×

Efficient Bayesian regularization for graphical model selection. (English) Zbl 1416.62317

Summary: There has been an intense development in the Bayesian graphical model literature over the past decade; however, most of the existing methods are restricted to moderate dimensions. We propose a novel graphical model selection approach for large dimensional settings where the dimension increases with the sample size, by decoupling model fitting and covariance selection. First, a full model based on a complete graph is fit under a novel class of mixtures of inverse-Wishart priors, which induce shrinkage on the precision matrix under an equivalence with Cholesky-based regularization, while enabling conjugate updates. Subsequently, a post-fitting model selection step uses penalized joint credible regions to perform model selection. This allows our methods to be computationally feasible for large dimensional settings using a combination of straightforward Gibbs samplers and efficient post-fitting inferences. Theoretical guarantees in terms of selection consistency are also established. Simulations show that the proposed approach compares favorably with competing methods, both in terms of accuracy metrics and computation times. We apply this approach to a cancer genomics data example.

MSC:

62H20 Measures of association (correlation, canonical correlation, etc.)
62F15 Bayesian inference
62P10 Applications of statistics to biology and medical sciences; meta analysis

Software:

HdBCS; glasso
PDFBibTeX XMLCite
Full Text: DOI Euclid

References:

[1] Ambros, V. (2004). “The functions of animal microRNAs.” Nature, 431, 350-355.
[2] Atay-Kayis, A. and Massam, H. (2005). “A Monte-Carlo Method for Computing the Marginal Likelihood in Nondecomposable Gaussian Graphical Models.” Biometrika, 92, 317-335. · Zbl 1094.62028 · doi:10.1093/biomet/92.2.317
[3] Baladandayuthapani, V., Ji. Y., Talluri, R., Nieto-Barajas, L. E. and Morris, J. S. (2010). “Bayesian Random Segmentation Models to Identify Shared Copy Number Aberrations for Array CGH Data.” Journal of American Statistical Association, 105, 1358-1375. · Zbl 1388.62312 · doi:10.1198/jasa.2010.ap09250
[4] Bondell, H. D. and Reich, B. J. (2012). “Consistent high-dimensional Bayesian variable selection via penalized credible regions.” Journal of the American Statistical Association, 107, 1610-1624. · Zbl 1258.62026 · doi:10.1080/01621459.2012.716344
[5] Bickel, P. J. and Levina, E. (2008). “Covariance regularization by thresholding”. Annals of Statistics, 36(6), 2577-2604. · Zbl 1196.62062 · doi:10.1214/08-AOS600
[6] Cai, T. and Liu, W. (2012). “Adaptive Thresholding for Sparse Covariance Matrix Estimation”. Journal of the American Statistical Association, 106, 672-684. · Zbl 1232.62086 · doi:10.1198/jasa.2011.tm10560
[7] Cancer Genome Atlas Research Network (2008). “Comprehensive genomic characterization defines human glioblastoma genes and core pathways.” Nature, 455, 1061-8.
[8] Carvalho, C. M., Polson, N. G., and Scott, J. G. (2009). “Handling sparsity via the horseshoe.” Journal of Machine Learning Research W&CP, 5, 73-80.
[9] Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010). “The horseshoe estimator for sparse signals.” Biometrika, 97, 465-480. · Zbl 1406.62021 · doi:10.1093/biomet/asq017
[10] Carvalho, C. M. and Scott, J. G. (2009). “Objective Bayesian model selection in Gaussian graphical models.” Biometrika, 96(3), 497-512. · Zbl 1170.62020 · doi:10.1093/biomet/asp017
[11] Chan, J. C. and Jeliazkov, I. (2009). “Estimation of Restricted Covariance Matrices.” Journal of the Computational and Graphical Statistics, 18(2), 457-480.
[12] Dawid, A. P. and Lauritzen, S. L. (1993). “Hyper markov Laws in the Statistical Analysis of Decomposable Graphical Models.” Annals of Statistics, 21, 1272-1317. · Zbl 0815.62038 · doi:10.1214/aos/1176349260
[13] Dellaportas, P., Giudici, P. and Roberts, G. (2003). “Bayesian inference for non-decomposable graphical Gaussian models.” Sankhyā, 65, 43-55. · Zbl 1192.62090
[14] Delfino K. R., Serão, N. V., Southey, B. R., Rodriguez-Zas, S. L. (2011). “Therapy-, gender- and race-specific microRNA markers, target genes and networks related to glioblastoma recurrence and survival.” Cancer Genomics Proteonomics, 8, 173-183.
[15] Dempster, A. P. (1972). “Covariance Selection.” Biometrics. 28, 157-175.
[16] Diaconis, P. and Ylvisaker, D. (1979). “Conjugate Priors for Exponential Families.” Annals of Statistics, 7, 269-281. · Zbl 0405.62011 · doi:10.1214/aos/1176344611
[17] Dong, H., Luo, L., Hong, S., Siu, H., Xiao, Y., Jin, L., Chen, R., and Xiong, M. (2010). “Integrated analysis of mutations, miRNA and mRNA expression in glioblastoma.” BMC Systems Biology, 4, 163.
[18] Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). “Least angle regression.” The Annals of Statistics, 32, 407-499. · Zbl 1091.62054 · doi:10.1214/009053604000000067
[19] Friedman, J., Hastie, T., and Tibshirani, R. (2008). “Sparse inverse covariance estimation with the graphical lasso.” Biostatistics, 9, 432-441. · Zbl 1143.62076 · doi:10.1093/biostatistics/kxm045
[20] Fitch, M. A., Jones, M. B., and Massam, H. (2014). “The Performance of Covariance Selection Methods That Consider Decomposable Models Only.” Bayesian Analysis, 9, 659-684. · Zbl 1327.62389 · doi:10.1214/14-BA874
[21] Fouskakis, D.; Ntzoufras, I.; Draper, D. (2009). “Bayesian variable selection using cost-adjusted BIC, with application to cost-effective measurement of quality of health care.” Annals Applied Statistics 3, 663-690. · Zbl 1166.62082 · doi:10.1214/08-AOAS207
[22] Frühwirth-Schnatter, S. and Tüchler, R. (2008). “Bayesian parsimonious covariance estimation for hierarchical linear mixed models.” Statistics and Computing, 18, 1-13.
[23] George, E. I. and McCulloch, R. (1993). “Variable Selection via Gibbs Sampling.” Journal of the American Statistical Association, 88, 881-889.
[24] Giudici, P. and Green, P. J. (1999). “Decomposable Graphical Gaussian Model Determination.” Biometrika, 86, 785-801. · Zbl 0940.62019 · doi:10.1093/biomet/86.4.785
[25] Green, P. J. and Thomas, A. (2013). “Sampling decomposable graphs using a Markov chain on junction trees.” Biometrika, 100, 91-110. · Zbl 1284.62172 · doi:10.1093/biomet/ass052
[26] Hahn, P. R. and Carvalho, C. M. (2015). “Decoupling Shrinkage and Selection in Bayesian Linear Models: A Posterior Summary Perspective.” Journal of the American Statistical Association, 110, 435-448. · Zbl 1373.62036 · doi:10.1080/01621459.2014.993077
[27] Herranz, H. and Cohen, S. M. (2010). “MicroRNAs and gene regulatory networks: managing the impact of noise in biological systems.” Genes and Development, 24, 1339-44.
[28] Huang, A. and Wand, M. P. (2013). “Simple marginally non-informative prior distributions for covariance matrices.” Bayesian Analysis, 8, 439-452. · Zbl 1329.62135 · doi:10.1214/13-BA815
[29] Jones, B., Carvalho, C., Dobra, A., Hans, C., Carter, C., and West, M. (2005). “Experiments in Stochastic Computation for High-dimensional Graphical Models.” Statistical Science, 20, 388-400. · Zbl 1130.62408 · doi:10.1214/088342305000000304
[30] Kundu, S. and Dunson, D. B. (2014). “Bayes variable selection in semi-parametric linear models.” Journal of the American Statistical Association, 109, 437-447. · Zbl 1367.62069 · doi:10.1080/01621459.2014.881153
[31] Kundu, S., Mallick, B. K., and Baladandayuthapani, V. (2018). “Supplementary Materials for “Efficient Bayesian Regularization for Graphical Model Selection”.” Bayesian Analysis. · Zbl 1416.62317
[32] Lee, S. T., Chu, K., Oh, H. J., Im, W. S., Lim, J. Y., Kim, S. K., Park, C. K., Jung, K. H., Lee, S. K., Kim, M., and Roh, J. K. (2011). “Let-7 microRNA inhibits the proliferation of human glioblastoma cells”, Journal of Neuro-Oncology, 102, 19-24.
[33] Lenkoski, A. and Dobra, A. (2011). “Computational aspects related to inference in Gaussian graphical models with the G-Wishart prior.” Journal of Computational and Graphical Statistics, 20, 140-157.
[34] Lewis, B. P., Burge, C. B., Bartel, D. P. (2005). “Conserved Seed Pairing, Often Flanked by Adenosines, Indicates that Thousands of Human Genes are MicroRNA Targets.” Cell, 120, 15-20.
[35] Lv, J. and Fan, Y. (2009). “A unified approach to model selection and sparse recovery using regularized least squares.” Annals of Statistics, 37, 3498-3528. · Zbl 1369.62156 · doi:10.1214/09-AOS683
[36] Meinshausen, N. and Bühlmann, P. (2006). “High-dimensional Graphs and Variable Selection with the Lasso”, Annals of Statistics, 34, 1436-1462. · Zbl 1113.62082 · doi:10.1214/009053606000000281
[37] Mohammadi, A. and Wit, E. C. (2015). “Bayesian Structure Learning in Sparse Gaussian Graphical Models.” Bayesian Analysis, 10, 109-138. · Zbl 1335.62056 · doi:10.1214/14-BA889
[38] Monti, R. P., Hellyer, P., Sharp, D., Leech, R., Anagnostopoulos, C., Montana, G. (2014). “Estimating time-varying brain connectivity networks from functional MRI time series.” NeuroImage, 103, 427-443.
[39] Morris, J. S., Brown, P. J., Herrick, R. C., Baggerly, K. A. and Coombes, K. R. (2008). “Bayesian analysis of mass spectrometry proteomic data using wavelet-based functional mixed models.” Biometrics, 64, 479-489. · Zbl 1137.62399 · doi:10.1111/j.1541-0420.2007.00895.x
[40] Newton, M. A., Noueiry, A., Sarkar, D. and Ahlquist, P. (2004). “Detecting differential gene expression with a semiparametric hierarchical mixture method.” Biostatistics, 5, 155-176. · Zbl 1096.62124 · doi:10.1093/biostatistics/5.2.155
[41] Peng, J., Wang, P., Zhou, N., and Zhu, J. (2009). “Partial Correlation Estimation by Joint Sparse Regression Models.” Journal of the American Statistical Association, 104, 735-746. · Zbl 1388.62046 · doi:10.1198/jasa.2009.0126
[42] Polson, N. G. and Scott, J. (2011). “Shrink globally, act locally: sparse Bayesian regularization and prediction.” In Bayesian Statistics 9 (J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West, eds.), pages 501-538. Oxford University Press, New York.
[43] Pourahmadi, M. (1999). “Joint mean-covariance models with applications to longitudinal data: unconstrained parameterisation.” Biometrika, 86, 677-690. · Zbl 0949.62066 · doi:10.1093/biomet/86.3.677
[44] Roverato, A. (2000). “Cholesky decomposition of a hyper inverse Wishart matrix.” Biometrika, 87, 99-112. · Zbl 0974.62047 · doi:10.1093/biomet/87.1.99
[45] Scott, J. G. and Carvalho, C. M. (2008). “Feature-Inclusion Stochastic Search for Gaussian Graphical Models.” Journal of Computational and Graphical Statistics, 17, 790-808.
[46] Smith, M. and Kohn, R. (2002). “Parsimonious covariance matrix estimation for longitudinal data.” Journal of the American Statistical Association, 97, 1141-1153. · Zbl 1041.62044 · doi:10.1198/016214502388618942
[47] Tang, W., Duan, J., Zhang, J. G., and Wang, Y. P. (2013). “Subtyping glioblastoma by combining miRNA and mRNA expression data using compressed sensing-based approach.” EURASIP Journal on Bioinformatics and Systems Biology, 2.
[48] Tibshirani, R. J. (2013). “The lasso problem and uniqueness.” Electronic Journal of Statistics, 7, 1456-1490. · Zbl 1337.62173 · doi:10.1214/13-EJS815
[49] Wang, H. (2012). “Bayesian Graphical Lasso Models and Efficient Posterior Computation.” Bayesian Analysis, 7, 771-790.
[50] Wang, H. and West, M. (2009). “Bayesian analysis of matrix normal graphical models.” Biometrika, 96, 821-834. · Zbl 1179.62042 · doi:10.1093/biomet/asp049
[51] Wong, A. J., Ruppert, J. M., Bigner, S. H., Grzeschik, C. H., Humphrey, P. A., Bigner, D. S., and Vogelstein, B. (1992). “Structural alterations of the epidermal growth factor receptor gene in human gliomas.” Proceedings of the National Academy of Sciences of the United States of America, 89, 2965-2969.
[52] Wong, F., Carter, C., and Kohn, R. (2003). “Efficient Estimation of Covariance Selection Models.” Biometrika, 90, 809-830. · Zbl 1436.62346 · doi:10.1093/biomet/90.4.809
[53] Wu, W. B. and Pourahmadi, M. (2003). “Nonparametric estimation of large covariance matrices of longitudinal data.” Biometrika, 90, 831-44. · Zbl 1436.62347 · doi:10.1093/biomet/90.4.831
[54] Yuan, M. and Lin, Y. (2007). “Model selection and estimation in the Gaussian graphical model.” Biometrika, 94, 19-35. · Zbl 1142.62408 · doi:10.1093/biomet/asm018
[55] Zou, H. and Li, R. (2008). “One-step sparse estimates in nonconcave penalized likelihood models (with discussion).” Annals of Statistics, 36, 1509-1566. · Zbl 1282.62112 · doi:10.1214/07-AOS0316REJ
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.