×

Sparse covariance estimation in heterogeneous samples. (English) Zbl 1274.62207

Summary: Standard Gaussian graphical models implicitly assume that the conditional independence among variables is common to all observations in the sample. However, in practice, observations are usually collected from heterogeneous populations where such an assumption is not satisfied, leading in turn to nonlinear relationships among variables. To address such situations we explore mixtures of Gaussian graphical models; in particular, we consider both infinite mixtures and infinite hidden Markov models where the emission distributions correspond to Gaussian graphical models. Such models allow us to divide a heterogeneous population into homogenous groups, with each cluster having its own conditional independence structure. As an illustration, we study the trends in foreign exchange rate fluctuations in the pre-Euro era.

MSC:

62F15 Bayesian inference
62H25 Factor analysis and principal components; correspondence analysis
62H30 Classification and discrimination; cluster analysis (statistical aspects)
62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62P20 Applications of statistics to economics

Software:

HdBCS
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Antoniak, C. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems., Annals of Statistics 2 , 1152-1174. · Zbl 0335.60034 · doi:10.1214/aos/1176342871
[2] Armstrong, H., Carter, C. K., Wong, K. F. & Kohn, R. (2009). Bayesian covariance matrix estimation using a mixture of decomposable graphical models., Statistics and Computing 19 , 303-316. · doi:10.1007/s11222-008-9093-8
[3] Atay-Kayis, A. & Massam, H. (2005). A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models., Biometrika 92 , 317-35. · Zbl 1094.62028 · doi:10.1093/biomet/92.2.317
[4] Beal, M. J., Ghahramani, Z. & Rasmussen, C. E. (2001). The infinite hidden markov model. In, Proceedings of Fourteenth Annual Conference on Neural Information Processing Systems .
[5] Bedford, T. & Cooke, R. M. (2002). Vines - a new graphical model for dependent random variables., Annals of Statistics 30 , 1031-1068. · Zbl 1101.62339 · doi:10.1214/aos/1031689016
[6] Berger, J. O. & Molina, G. (2005). Posterior model probabilities via path-based pairwise priors., Statistica Neerlandica 59 , 3-15. · Zbl 1069.62021 · doi:10.1111/j.1467-9574.2005.00275.x
[7] Blackwell, D. & MacQueen, J. B. (1973). Ferguson distribution via Pólya urn schemes., The Annals of Statistics 1 , 353-355. · Zbl 0276.62010 · doi:10.1214/aos/1176342372
[8] Cappé, O., Moulines, E. & Ryden, T. (2005)., Inference in Hidden Markov Models . Springer. · Zbl 1080.62065
[9] Carvalho, C. M., Massam, H. & West, M. (2007). Simulation of hyper-inverse Wishart distributions in graphical models., Biometrika 94 , 647-659. · Zbl 1135.62011 · doi:10.1093/biomet/asm056
[10] Carvalho, C. M. & West, M. (2007). Dynamic matrix-variate graphical models., Bayesian Analysis 2 , 69-98. · Zbl 1331.62040 · doi:10.1214/07-BA204
[11] Castelo, R. & Roverato, A. (2006). A robust procedure for Gaussian graphical model search from microarray data with p larger than n., Journal of Machine Learning Research 7 , 2621-2650. · Zbl 1222.68158
[12] Dawid, A. P. & Lauritzen, S. L. (1993). Hyper Markov laws in the statistical analysis of decomposable graphical models., Annals of Statistics 21 , 1272-1317. · Zbl 0815.62038 · doi:10.1214/aos/1176349260
[13] Dempster, A. P. (1972). Covariance selection., Biometrics 28 , 157-75.
[14] Diaconnis, P. & Ylvisaker, D. (1979). Conjugate priors for exponential families., Annals of Statistics 7 , 269-81. · Zbl 0405.62011 · doi:10.1214/aos/1176344611
[15] Dobra, A., Eicher, T. & Lenkoski, A. (2010). Modeling uncertainty in macroeconomic growth determinants using gaussian graphical models., Statistical Methodology 7 , 292-306. · Zbl 05898179 · doi:10.1016/j.stamet.2009.11.003
[16] Dobra, A., Hans, C., Jones, B., Nevins, J. R., Yao, G. & West, M. (2004). Sparse graphical models for exploring gene expression data., Journal of Multivariate Analysis 90 , 196-212. · Zbl 1047.62104 · doi:10.1016/j.jmva.2004.02.009
[17] Dobra, A., Lenkoski, A. & Rodríguez, A. (2011). Bayesian inference for general Gaussian graphical models with application to multivariate lattice data., Journal of the American Statistical Association · Zbl 1234.62018
[18] Escobar, M. D. & West, M. (1995). Bayesian density estimation and inference using mixtures., Journal of the American Statistical Association 90 , 577-588. · Zbl 0826.62021 · doi:10.2307/2291069
[19] Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems., Annals of Statistics 1 , 209-230. · Zbl 0255.62037 · doi:10.1214/aos/1176342360
[20] Ferguson, T. S. (1974). Prior distributions on spaces of probability measures., Annals of Statistics 2 , 615-629. · Zbl 0286.62008 · doi:10.1214/aos/1176342752
[21] Fraley, C. & Raftery, A. E. (2007). Bayesian regularization for normal mixture estimation and model-based clustering., Journal of Classification 24 , 155-181. · Zbl 1159.62302 · doi:10.1007/s00357-007-0004-5
[22] Friedman, N. (2004). Inferring cellular networks using probabilistic graphical models., Science 6 , 799-805.
[23] van Gael, J., Saatci, Y., Teh, Y.-W. & Ghahramani, Z. (2008). Beam sampling for the infinite hidden markov model. In, Proceedings of the 25th International Conference on Machine Learning (ICML) .
[24] Green, P. & Richardson, S. (2001). Modelling heterogeneity with and without the Dirichlet process., Scandinavian Journal of Statistics 28 , 355-375. · Zbl 0973.62031 · doi:10.1111/1467-9469.00242
[25] Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination., Biometrika 82 . · Zbl 0861.62023 · doi:10.1093/biomet/82.4.711
[26] Guo, J., Levina, E., Michailidis, G. & Zhu (2011). Joint estimation of multiple graphical models., Biometrika 98 , 1-15. · Zbl 1214.62058 · doi:10.1093/biomet/asq060
[27] Heinz, D. (2009). Building hyper Dirichlet processes for gaphical models., Electronic Journal of Statistics 3 , 290-315. · Zbl 1326.62012 · doi:10.1214/08-EJS269
[28] Ishwaran, H. & James, L. F. (2001). Gibbs sampling methods for stick-breaking priors., Journal of the American Statistical Association 96 , 161-173. · Zbl 1014.62006 · doi:10.1198/016214501750332758
[29] Ishwaran, H. & Zarepour, M. (2002). Dirichlet prior sieves in finite normal mixtures., Statistica Sinica 12 , 941-963. · Zbl 1002.62028
[30] Jain, S. & Neal, R. M. (2004). A split-merge Markov chain Monte Carlo procedure for the Dirichlet process mixture model., Journal of Graphical and Computational Statistics 13 , 158-182. · doi:10.1198/1061860043001
[31] Jain, S. & Neal, R. M. (2007). Splitting and merging components of a nonconjugate dirichlet process mixture model., Bayesian Analysis 2 , 445-472. · Zbl 1331.62145 · doi:10.1214/07-BA219
[32] Jones, B., Carvalho, C., Dobra, A., Hans, C., Carter, C. & West, M. (2005). Experiments in stochastic computation for high-dimensional graphical models., Statististical Science 20 , 388-400. · Zbl 1130.62408 · doi:10.1214/088342305000000304
[33] Lau, J. W. & Green, P. (2007). Bayesian model based clustering procedures., Journal of Computational and Graphical Statistics 16 , 526-558. · doi:10.1198/106186007X238855
[34] Lauritzen, S. L. (1996)., Graphical Models . Oxford University Press. · Zbl 0907.62001
[35] Lee, J., Müller, P., Trippa, L. & Quintana, F. A. (2009). Defining predictive probability functions for species sampling models. Technical report, Pontificia Universidad Católica de, Chile. · Zbl 1331.62152
[36] Lenkoski, A. & Dobra, A. (2011). Computational aspects related to inference in Gaussian graphical models with the G-wishart prior., Journal of Computational and Graphical Statistics 20 , 140-157. · Zbl 1232.62046 · doi:10.1198/jcgs.2010.08181
[37] Letac, G. & Massam, H. (2007). Wishart distributions for decomposable graphs., Annals of Statistics 35 , 1278-323. · Zbl 1194.62078 · doi:10.1214/009053606000001235
[38] Liu, J. S., Liang, F. & Wong, W. H. (2000). The use of multiple-try method and local optimization in metropolis sampling., Journal of the American Statistical Association 95 , 121-134. · Zbl 1072.65505 · doi:10.2307/2669532
[39] Lo, A. Y. (1984). On a class of Bayesian nonparametric estimates: I. Density estimates., Annals of Statistics 12 , 351-357. · Zbl 0557.62036 · doi:10.1214/aos/1176346412
[40] Muirhead, R. J. (2005)., Aspects of Multivariate Statistical Theory . John Wiley & Sons. · Zbl 0556.62028
[41] Müller, P., Erkanli, A. & West, M. (1996). Bayesian curve fitting using multivariate normal mixtures., Biometrika 83 , 67-79. · Zbl 0865.62029 · doi:10.1093/biomet/83.1.67
[42] Müller, P., Quintana, F. & Rosner, G. (2004). Hierarchical meta-analysis over related non-parametric Bayesian models., Journal of the Royal Statistical Society, Series B 66 , 735-749. · Zbl 1046.62053 · doi:10.1111/j.1467-9868.2004.05564.x
[43] Neal, R. M. (2000). Markov chain sampling methods for Dirichlet process mixture models., Journal of Computational and Graphical Statistics 9 , 249-265.
[44] Ongaro, A. & Cattaneo, C. (2004). Discrete random probability measures: a general framework for nonparametric Bayesian inference., Statistics and Probability Letters 67 , 33-45. · Zbl 1075.62041
[45] Pitman, J. (1995). Exchangeable and partially exchangeable random partitions., Probability Theory and Related Fields 102 , 145-158. · Zbl 0821.60047 · doi:10.1007/BF01213386
[46] Quintana, F. & Iglesias, P. L. (2003). Bayesian clustering and product partition models., Journal of the Royal Statistical Society, Series B. 65 , 557-574. · Zbl 1065.62115 · doi:10.1111/1467-9868.00402
[47] Roberts, G. & Papaspiliopoulos, O. (2008). Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models., Biometrika 95 , 169-186. · Zbl 1437.62576 · doi:10.1093/biomet/asm086
[48] Rodríguez, A., Dunson, D. B. & Gelfand, A. E. (2009). Bayesian nonparametric functional data analysis through density estimation., Biometrika 96 , 149-162. · Zbl 1163.62027 · doi:10.1093/biomet/asn054
[49] Rodríguez, A. & Vuppala, R. (2009). Probabilistic classification using Bayesian nonparametric mixture models. Technical report, University of California, Santa, Cruz.
[50] Roverato, A. (2002). Hyper inverse Wishart distribution for non-decomposable graphs and its application to Bayesian inference for Gaussian graphical models., Scandinavian Journal of Statistics 29 , 391-411. · Zbl 1036.62027 · doi:10.1111/1467-9469.00297
[51] Scott, J. G. & Carvalho, C. M. (2008). Feature-inclusion stochastic search for Gaussian graphical models., Journal of Computational and Graphical Statistics 17 , 790-808. · doi:10.1198/106186008X382683
[52] Sethuraman, J. (1994). A constructive definition of Dirichelt priors., Statistica Sinica 4 , 639-650. · Zbl 0823.62007
[53] Stephens, M. (2000). Dealing with label switching in mixture models., Journal of the Royal Statistical Society, Series B. 62 , 795-809. · Zbl 0957.62020 · doi:10.1111/1467-9868.00265
[54] Teh, Y. W., Jordan, M. I., Beal, M. J. & Blei, D. M. (2006). Sharing clusters among related groups: Hierarchical Dirichlet processes., Journal of the American Statistical Association 101 , 1566-1581. · Zbl 1171.62349 · doi:10.1198/016214506000000302
[55] Thiesson, B., Meek, C., Chickering, D. M. & Heckerman, D. (1997). Learning mixtures of DAG models. In, Proceedings of the Conference on Uncertainty in Artificial Intelligence , pp. 504-513. Morgan Kaufmann, Inc. · Zbl 0974.62027
[56] Wainwright, M. J., Ravikumar, P. & Lafferty, J. D. (2006). High-dimensional graphical model selection using, \ell 1 -regularized logistic regression. In In Neural Information Processing Systems . MIT Press. · Zbl 1189.62115
[57] Walker, S. G. (2007). Sampling the dirichlet mixture model with slices., Communications in Statistics - Simulation and Computation 36 , 45-54. · Zbl 1113.62058 · doi:10.1080/03610910601096262
[58] Wang, H. & Carvalho, C. M. (2010). Simulation of hyper-inverse Wishart distributions for non-decomposable graphs., Electronic Journal of Statistics 4 , 1470-1475. · Zbl 1329.60008 · doi:10.1214/10-EJS591
[59] Wang, H., Reeson, C. & Carvalho, C. M. (2011). Dynamic Financial Index Models: Modeling Conditional Dependences via Graphs., Bayesian Analysis · Zbl 1330.91187 · doi:10.1214/11-BA624
[60] Wang, H. & West, M. (2009). Bayesian analysis of matrix normal graphical models., Biometrika 96 , 821-834. · Zbl 1179.62042 · doi:10.1093/biomet/asp049
[61] West, M., Blanchette, H., Dressman, H., Huang, E., Ishida, S., Spang, R., Zuzan, H., Olson, J. A., Marks, J. R. & Nevings, J. R. (2001). Predicting the clinical status of human breast cancer by using gne expression profiles., Proceedings of the National Academi of Sciences 98 , 11462-11467.
[62] West, M. & Harrison, J. (1997)., Bayesian Forecasting and Dynamic Models . Springer - Verlag, New York, 2nd edition. · Zbl 0871.62026
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.