×

High-dimensional Ising model selection with Bayesian information criteria. (English) Zbl 1309.62050

Summary: We consider the use of Bayesian information criteria for selection of the graph underlying an Ising model. In an Ising model, the full conditional distributions of each variable form logistic regression models, and variable selection techniques for regression allow one to identify the neighborhood of each node and, thus, the entire graph. We prove high-dimensional consistency results for this pseudo-likelihood approach to graph selection when using Bayesian information criteria for the variable selection problems in the logistic regressions. The results pertain to scenarios of sparsity, and following related prior work the information criteria we consider incorporate an explicit prior that encourages sparsity.

MSC:

62F12 Asymptotic properties of parametric estimators
62J12 Generalized linear models (logistic models)

Software:

mboost; glmnet
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Anandkumar, A., Tan, V. Y. F., Huang, F. and Willsky, A. S. (2012). High-dimensional structure estimation in Ising models: Local separation criterion., Ann. Statist. 40 1346-1375. · Zbl 1297.62124 · doi:10.1214/12-AOS1009
[2] Besag, J. E. (1972). Nearest-neighbour systems and the auto-logistic model for binary data., J. Roy. Statist. Soc. Ser. B 34 75-83. · Zbl 0268.60088
[3] Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems., J. Roy. Statist. Soc. Ser. B 36 192-236. With discussion by D. R. Cox, A. G. Hawkes, P. Clifford, P. Whittle, K. Ord, R. Mead, J. M. Hammersley, and M. S. Bartlett and with a reply by the author. · Zbl 0327.60067
[4] Bogdan, M., Ghosh, J. K. and Doerge, R. W. (2004). Modifying the Schwarz Bayesian information criterion to locate multiple interacting quantitative trait loci., Genetics 167 989-999.
[5] Broman, K. W. and Speed, T. P. (2002). A model selection approach for the identification of quantitative trait loci in experimental crosses., J. R. Stat. Soc. Ser. B Stat. Methodol. 64 641-656. · Zbl 1067.62108 · doi:10.1111/1467-9868.00354
[6] Bühlmann, P. and van de Geer, S. (2011)., Statistics for high-dimensional data . Springer Series in Statistics . Springer, Heidelberg. Methods, theory and applications. · Zbl 1273.62015 · doi:10.1007/978-3-642-20192-9
[7] Chen, J. and Chen, Z. (2008). Extended Bayesian information criteria for model selection with large model spaces., Biometrika 95 759-771. · Zbl 1437.62415 · doi:10.1093/biomet/asn034
[8] Chen, J. and Chen, Z. (2012). Extended BIC for small-\(n\)-large-\(P\) sparse GLM., Statist. Sinica 22 555-574. · Zbl 1238.62080 · doi:10.5705/ss.2010.216
[9] Foygel, R. (2012). Prediction and model selection for high-dimensional data with sparse or low-rank structure. PhD thesis, The University of, Chicago.
[10] Foygel, R. and Drton, M. (2011). Bayesian model choice and information criteria in sparse generalized linear models., ArXiv e-prints 1112.5635.
[11] Foygel, R. and Drton, M. (2014). High-dimensional Ising model selection with Bayesian information criteria., ArXiv e-prints 1403.3374v1. · Zbl 1309.62050 · doi:10.1214/15-EJS1012
[12] Friedman, J., Hastie, T. and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent., Journal of Statistical Software 33 1-22.
[13] Frommlet, F., Ruhaltinger, F., Twaróg, P. and Bogdan, M. (2012). Modified versions of Bayesian information criterion for genome-wide association studies., Comput. Statist. Data Anal. 56 1038-1051.
[14] Höfling, H. and Tibshirani, R. (2009). Estimation of sparse binary pairwise Markov networks using pseudo-likelihoods., J. Mach. Learn. Res. 10 883-906. · Zbl 1245.62121
[15] Hothorn, T., Bühlmann, P., Kneib, T., Schmid, M. and Hofner, B. (2013). mboost: Model-based boosting., R package version 2.2-3 .
[16] Jalali, A., Johnson, C. C. and Ravikumar, P. K. (2011). On learning discrete graphical models using greedy methods. In, Advances in Neural Information Processing Systems 1935-1943.
[17] Kindermann, R. and Snell, J. L. (1980)., Markov random fields and their applications . Contemporary Mathematics 1 . American Mathematical Society, Providence, R.I. · Zbl 1229.60003
[18] Koltchinskii, V. (2011)., Oracle inequalities in empirical risk minimization and sparse recovery problems . Lecture Notes in Mathematics 2033 . Springer, Heidelberg. Lectures from the 38th Probability Summer School held in Saint-Flour, 2008, École d’Été de Probabilités de Saint-Flour. [Saint-Flour Probability Summer School]. · Zbl 1223.91002 · doi:10.1007/978-3-642-22147-7
[19] Lauritzen, S. L. (1996)., Graphical models . Oxford Statistical Science Series 17 . The Clarendon Press Oxford University Press, New York. Oxford Science Publications. · Zbl 1273.62015 · doi:10.1007/978-3-642-20192-9
[20] Loh, P.-L. and Wainwright, M. J. (2013). Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses., Ann. Statist. 41 3022-3049. · Zbl 1288.62081 · doi:10.1214/13-AOS1162
[21] Lorentz, G. G., Golitschek, M. v. and Makovoz, Y. (1996)., Constructive approximation . Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 304 . Springer-Verlag, Berlin. Advanced problems. · Zbl 0910.41001
[22] Luo, S. and Chen, Z. (2013). Selection consistency of EBIC for GLIM with non-canonical links and diverging number of parameters., Stat. Interface 6 275-284. · Zbl 1327.62429 · doi:10.4310/SII.2013.v6.n2.a10
[23] Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso., Ann. Statist. 34 1436-1462. · Zbl 1113.62082 · doi:10.1214/009053606000000281
[24] Meinshausen, N. and Bühlmann, P. (2010). Stability selection., J. R. Stat. Soc. Ser. B Stat. Methodol. 72 417-473. · doi:10.1111/j.1467-9868.2010.00740.x
[25] Menne, M. J., Williams Jr., C. N. and Vose, R. S. (2011). United States Historical Climatology Network Daily Temperature, Precipitation, and Snow, Data.
[26] Ravikumar, P., Wainwright, M. J. and Lafferty, J. D. (2010). High-dimensional Ising model selection using \(\ell_1\)-regularized logistic regression., Ann. Statist. 38 1287-1319. · Zbl 1189.62115 · doi:10.1214/09-AOS691
[27] Roudi, Y., Aurell, E. and Hertz, J. A. (2009). Statistical physics of pairwise probability models., Frontiers in Computational Neuroscience 3 .
[28] Santhanam, N. P. and Wainwright, M. J. (2012). Information-theoretic limits of selecting binary graphical models in high dimensions., IEEE Transactions on Information Theory 58 4117-4134. · Zbl 1365.62212 · doi:10.1109/TIT.2012.2191659
[29] Schwarz, G. (1978). Estimating the dimension of a model., Ann. Statist. 6 461-464. · Zbl 0379.62005 · doi:10.1214/aos/1176344136
[30] Scott, J. G. and Berger, J. O. (2010). Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem., Ann. Statist. 38 2587-2619. · Zbl 1200.62020 · doi:10.1214/10-AOS792
[31] Shorack, G. R. (2000)., Probability for statisticians . Springer Texts in Statistics . Springer-Verlag, New York. · Zbl 0951.62005 · doi:10.1007/b98901
[32] Żak-Szatkowska, M. and Bogdan, M. (2011). Modified versions of the Bayesian information criterion for sparse generalized linear models., Comput. Statist. Data Anal. 55 2908-2924. · Zbl 1218.62073 · doi:10.1016/j.csda.2011.04.016
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.