×

zbMATH — the first resource for mathematics

Efficient computation for differential network analysis with applications to quadratic discriminant analysis. (English) Zbl 07160696
Summary: Differential network analysis is an important statistical problem with wide applications. Many statisticians focus on binary problems and propose to perform such analysis by obtaining sparse estimates of the difference between precision matrices. These methods are supported by excellent theoretical properties and practical performance. However, efficient computation for these methods remains a challenging problem. A novel algorithm referred to as the SMORE algorithm is proposed for differential network analysis. The SMORE algorithm has low storage cost and high computation speed, especially in the presence of strong sparsity. In the meantime, the SMORE algorithm provides a unified framework for binary and multiple network problems. In addition, the SMORE algorithm can be applied in high-dimensional quadratic discriminant analysis problems as well, leading to a new approach for multiclass high-dimensional quadratic discriminant analysis. Numerical studies confirm the stability and the efficiency of the proposed SMORE algorithm in both differential network analysis and quadratic discriminant analysis.
MSC:
62 Statistics
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Bandyopadhyay, S.; Mehta, M.; Kuo, D.; Sung, M.-K.; Chuang, R.; Jaehnig, E. J.; Bodenmiller, B.; Licon, K.; Copeland, W.; Shales, M.; Fiedler, D.; Dutkowski, J.; Guénolé, A.; van Attikum, H.; Shokat, K. M.; Kolodner, R. D.; Huh, W.-K.; Aebersold, R.; Keogh, M.-C.; Krogan, N. J.; Ideker, T., Rewiring of genetic networks in response to DNA damage, Science, 330, 6009, 1385-1389 (2010)
[2] Banerjee, O.; El Ghaoui, L.; d’Aspremont, A., Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data, J. Mach. Learn. Res., 9, 485-516 (2008) · Zbl 1225.68149
[3] Barabási, A.-L.; Gulbahce, N.; Loscalzo, J., Network medicine: a network-based approach to human disease, Nat. Rev. Genet., 12, 1, 56 (2011)
[4] Barabasi, A.-L.; Oltvai, Z. N., Network biology: understanding the cell’s functional organization, Nat. Rev. Genet., 5, 2, 101 (2004)
[5] Basso, K.; Margolin, A.; Stolovitzky, G.; Klein, U.; Dalla-Favera, R.; Califano, A., Reverse engineering of regulatory networks in human B cells, Nature Genet., 37, 382-390 (2005)
[6] Beck, A.; Teboulle, M., A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2, 1, 183-202 (2009) · Zbl 1175.94009
[7] Bertsekas, D. P., Nonlinear Programming (2016), Athena Scientific · Zbl 1360.90236
[8] Boik, R. J., Spectral models for covariance matrices, Biometrika, 89, 1, 159-182 (2002) · Zbl 0997.62046
[9] Bonneau, R.; Facciotti, M.; Reiss, D.; Schmid, A.; Pan, M.; Kaur, A.; Thorsson, V.; Shannon, P.; Johnson, M.; Bare, J. C.; Longabaugh, W.; Vuthoori, M.; Whitehead, K.; Madar, A.; Suzuki, L.; Mori, T.; Chang, D.-E.; Diruggiero, J.; H Johnson, C.; Baliga, N., A predictive model for transcriptional control of physiology in a free living cell, Cell, 131, 1354-1365 (2008)
[10] Boyd, S.; Parikh, N.; Chu, E.; Peleato, B.; Eckstein, J., Distributed optimization and statistical learning via the alternating direction method of multipliers, Found. Trends Mach. Learn., 3, 1, 1-122 (2011) · Zbl 1229.90122
[11] Breiman, L., Random forests, Mach. Learn., 45, 1, 5-32 (2001) · Zbl 1007.68152
[12] Cai, T.; Liu, W., Adaptive thresholding for sparse covariance matrix estimation, J. Amer. Statist. Assoc., 106, 494, 672-684 (2011) · Zbl 1232.62086
[13] Cai, T. T.; Liu, W.; Luo, X., A constrained \(\ell_1\) minimization approach to sparse precision matrix estimation, J. Amer. Statist. Assoc., 106, 494, 594-607 (2011) · Zbl 1232.62087
[14] Candes, E.; Tao, T., The Dantzig selector: Statistical estimation when p is much larger than n, Ann. Statist., 35, 6, 2313-2351 (2007) · Zbl 1139.62019
[15] Chiquet, J.; Grandvalet, Y.; Ambroise, C., Inferring multiple graphical structures, Stat. Comput., 21, 4, 537-553 (2011) · Zbl 1221.62085
[16] Clemmensen, L.; Hastie, T.; Witten, D.; Ersbøll, B., Sparse discriminant analysis, Technometrics, 53, 4, 406-413 (2011)
[17] Cook, R. D.; Forzani, L., Covariance reducing models: An alternative to spectral modelling of covariance matrices, Biometrika, 95, 4, 799-812 (2008) · Zbl 1437.62428
[18] Cortes, C.; Vapnik, V., Support-vector networks, Mach. Learn., 20, 3, 273-297 (1995) · Zbl 0831.68098
[19] Dimitriadou, E.; Hornik, K.; Leisch, F.; Meyer, D.; Weingessel, A., E1071: Misc functions of the department of statistics (E1071), TU Wien (2009), R package version 1.5-24
[20] Efron, B.; Hastie, T.; Johnstone, I.; Tibshirani, R., Least angle regression, Ann. Statist., 32, 2, 407-499 (2004) · Zbl 1091.62054
[21] Fan, J.; Feng, Y.; Tong, X., A road to classification in high dimensional space: the regularized optimal affine discriminant, J. R. Stat. Soc. Ser. B Stat. Methodol., 74, 4, 745-771 (2012) · Zbl 1411.62167
[22] Fan, J.; Ke, Z. T.; Liu, H.; Xia, L., QUADRO: A supervised dimension reduction method via Rayleigh quotient optimization, Ann. Statist., 43, 4, 1498 (2015) · Zbl 1317.62054
[23] Flury, B. N., Common principal components in k groups, J. Amer. Statist. Assoc., 79, 388, 892-898 (1984)
[24] Flury, B. K., Two generalizations of the common principal component model, Biometrika, 74, 1, 59-69 (1987) · Zbl 0613.62076
[25] Franks, A.; Hoff, P., Shared subspace models for multi-group covariance estimation (2016), arXiv preprint arXiv:1607.03045
[26] Friedman, J.; Hastie, T.; Höfling, H.; Tibshirani, R., Pathwise coordinate optimization, Ann. Appl. Stat., 1, 2, 302-332 (2007) · Zbl 1378.90064
[27] Friedman, J.; Hastie, T.; Tibshirani, R., Sparse inverse covariance estimation with the graphical LASSO, Biostatistics (Oxford, England), 9, 432-441 (2008) · Zbl 1143.62076
[28] Friedman, J., Hastie, T., Tibshirani, R., 2009. In: The Elements of Statistical Learning. In: Springer Series in Statistics, vol. 1. New York, USA. · Zbl 1273.62005
[29] Friedman, J.; Hastie, T.; Tibshirani, R., Regularization paths for generalized linear models via coordinate descent, J. Stat. Softw., 33, 1, 1-22 (2010)
[30] Goldstein, T.; Osher, S., The split bregman method for L1-regularized problems, SIAM J. Imaging Sci., 2, 2, 323-343 (2009) · Zbl 1177.65088
[31] Guo, J.; Levina, E.; Michailidis, G.; Zhu, J., Joint estimation of multiple graphical models, Biometrika, 98, 1, 1-15 (2011) · Zbl 1214.62058
[32] Ha, M. J.; Baladandayuthapani, V.; Do, K.-A., DINGO: differential network analysis in genomics, Bioinformatics, 31, 21, 3413-3420 (2015) · Zbl 1335.46011
[33] Hudson, N. J.; Reverter, A.; Dalrymple, B. P., A differential wiring analysis of expression data correctly identifies the gene containing the causal mutation, PLoS Comput. Biol., 5, 840-845 (2009)
[34] Jiang, B.; Wang, X.; Leng, C., A direct approach for sparse quadratic discriminant analysis, J. Mach. Learn. Res., 19, 1, 1098-1134 (2018)
[35] Lam, C.; Fan, J., Sparsistency and rates of convergence in large covariance matrix estimation, Ann. Statist., 37, 6B, 4254-4278 (2009) · Zbl 1191.62101
[36] Ledoit, O.; Wolf, M., A well-conditioned estimator for large-dimensional covariance matrices, J. Multivariate Anal., 88, 2, 365-411 (2004) · Zbl 1032.62050
[37] Leiserson, M.; Vandin, F.; Wu, H.-T.; Dobson, J.; V Eldridge, J.; L Thomas, J.; Papoutsaki, A.; Kim, Y.; Niu, B.; Mclellan, M.; Lawrence, M.; Gonzalez-Perez, A.; Tamborero, D.; Cheng, Y.; A Ryslik, G.; L’øpez-Bigas, N.; Getz, G.; Ding, L.; Raphael, B., Pan-cancer network analysis identifies combinations of rare somatic mutations across pathways and protein complexes, Nature Genet. (2014)
[38] Li, Q.; Shao, J., Sparse quadratic discriminant analysis for high dimensional data, Statist. Sinica, 25, 2, 457-473 (2015) · Zbl 06503804
[39] Li, X.; Zhao, T.; Yuan, X.; Liu, H., The flare package for high dimensional linear regression and precision matrix estimation in R, J. Mach. Learn. Res., 16, 553-557 (2015) · Zbl 1337.62007
[40] Liaw, A.; Wiener, M., Package ’RandomForest’: Breiman and Cutler’s Random Forests for Classification and Regression, Vol. 4, 6-10 (2014), R Development Core Team
[41] Liu, W., Structural similarity and difference testing on multiple sparse Gaussian graphical models, Ann. Statist., 45, 6, 2680-2707 (2017) · Zbl 06838147
[42] Liu, W.; Luo, X., High-dimensional sparse precision matrix estimation via sparse column inverse operator (2012), arXiv preprint
[43] Mai, Q.; Yang, Y.; Zou, H., Multiclass sparse discriminant analysis, Statist. Sinica, 29, 97-111 (2019) · Zbl 1412.62081
[44] Mai, Q.; Zou, H.; Yuan, M., A direct approach to sparse discriminant analysis in ultra-high dimensions, Biometrika, 99, 1, 29-42 (2012) · Zbl 1437.62550
[45] Meier, L.; Van De Geer, S.; Bühlmann, P., The group lasso for logistic regression, J. R. Stat. Soc. Ser. B Stat. Methodol. (2007)
[46] Meinshausen, N.; Bühlmann, P., High-dimensional graphs and variable selection with the lasso, Ann. Statist., 34, 3, 1436-1462 (2006) · Zbl 1113.62082
[47] Mohan, K.; Chung, M.; Han, S.; Witten, D.; in Lee, S.; Fazel, M., Structured learning of Gaussian graphical models, (Pereira, F.; Burges, C. J.C.; Bottou, L.; Weinberger, K. Q., Advances in Neural Information Processing Systems, Vol. 25 (2012), Curran Associates, Inc.), 620-628
[48] Ortega, J. M.; Rheinboldt, W. C., Iterative Solution of Nonlinear Equations in Several Variables (2000), Society for Industrial and Applied Mathematics: Society for Industrial and Applied Mathematics Philadelphia, PA, USA · Zbl 0949.65053
[49] Pan, Y.; Mai, Q.; Zhang, X., Covariate-adjusted tensor classification in high-dimensions, J. Amer. Statist. Assoc., 114, 1305-1319 (2019) · Zbl 1428.62291
[50] Patrick, D.; Pei, W.; M., W. D., The joint graphical lasso for inverse covariance estimation across multiple classes, J. R. Stat. Soc. Ser. B Stat. Methodol., 76, 2, 373-397 (2013)
[51] Pereira-Leal, J. B.; Enright, A. J.; Ouzounis, C. A., Detection of functional modules from protein interaction networks, Proteins, 54, 1, 49-57 (2004)
[52] Schott, J. R., Partial common principal component subspaces, Biometrika, 86, 4, 899-908 (1999) · Zbl 0942.62066
[53] Sha, F.; Saul, L. K.; Lee, D. D., Multiplicative updates for nonnegative quadratic programming in support vector machines, (Becker, S.; Thrun, S.; Obermayer, K., Advances in Neural Information Processing Systems, Vol. 15 (2003), MIT Press), 1065-1072
[54] Shalev-Shwartz, S.; Tewari, A., Stochastic methods for L1-regularized loss minimization, J. Mach. Learn. Res., 12, 1865-1892 (2011) · Zbl 1280.62081
[55] Shao, J.; Wang, Y.; Deng, X.; Wang, S., Sparse linear discriminant analysis by thresholding for high dimensional data, Ann. Statist., 39, 2, 1241-1265 (2011) · Zbl 1215.62062
[56] Sun, T.; Zhang, C.-H., Sparse matrix inversion with scaled Lasso, J. Mach. Learn. Res., 14, 1, 3385-3418 (2013) · Zbl 1318.62184
[57] Tibshirani, R., Regression shrinkage and selection via the Lasso, J. R. Stat. Soc. Ser. B Stat. Methodol., 58, 267-288 (1996) · Zbl 0850.62538
[58] Tseng, P., Convergence of a block coordinate descent method for nondifferentiable minimization, J. Optim. Theory Appl., 109, 475-494 (2001) · Zbl 1006.65062
[59] Wang, W.; Zhang, X.; Li, L., Common reducing subspace model and network alternation analysis, Biometrics (2019)
[60] Wille, A.; Zimmermann, P.; Vranová, E.; Fürholz, A.; Laule, O.; Bleuler, S.; Hennig, L.; Prelić, A.; von Rohr, P.; Thiele, L.; Zitzler, E.; Gruissem, W.; Bühlmann, P., Sparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana, Genome Biol., 5, 11, R92 (2004)
[61] Witten, D. M.; Tibshirani, R.; Hastie, T., A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis, Biostatistics, 10, 3, 515-534 (2009)
[62] Wu, Y., An ordinary differential equation-based solution path algorithm, J. Nonparametr. Stat., 23, 1, 185-199 (2011) · Zbl 1359.62300
[63] Wu, T. T.; Lange, K., Coordinate descent algorithms for lasso penalized regression, Ann. Appl. Stat., 2, 1, 224-244 (2008) · Zbl 1137.62045
[64] Xu, P.; Zhu, J.; Zhu, L.; Li, Y., Covariance-enhanced discriminant analysis, Biometrica, 102, 1, 33-45 (2015) · Zbl 1345.62091
[65] Yang, J.; Zhang, Y., Alternating direction algorithms for \(\ell_1\)-problems in compressive sensing, SIAM J. Sci. Comput., 33, 1, 250-278 (2011) · Zbl 1256.65060
[66] Yin, W.; Osher, S.; Goldfarb, D.; Darbon, J., Bregman iterative algorithms for \(\ell_1\)-minimization with applications to compressed sensing, SIAM J. Imaging Sci., 1, 1, 143-168 (2008) · Zbl 1203.90153
[67] Yuan, M., High dimensional inverse covariance matrix estimation via linear programming, J. Mach. Learn. Res., 11, 2261-2286 (2010) · Zbl 1242.62043
[68] Yuan, M.; Lin, Y., Model selection and estimation in regression with grouped variables, J. R. Stat. Soc. Ser. B Stat. Methodol., 68, 1, 49-67 (2006) · Zbl 1141.62030
[69] Yuan, M.; Lin, Y., Model selection and estimation in the Gaussian graphical model, Biometrika, 94, 1, 19-35 (2007) · Zbl 1142.62408
[70] Yuan, H.; Xi, R.; Chen, C.; Deng, M., Differential network analysis via lasso penalized D-trace loss, Biometrika, 104, 4, 755-770 (2017) · Zbl 07072326
[71] Zhang, T.; Zou, H., Sparse precision matrix estimation via lasso penalized D-trace loss, Biometrika, 101, 1, 103-120 (2014) · Zbl 1285.62063
[72] Zhao, S. D.; Cai, T. T.; Li, H., Direct estimation of differential networks, Biometrika, 101, 2, 253-268 (2014) · Zbl 1452.62865
[73] Zhou, H.; Wu, Y., A generic path algorithm for regularized statistical estimation, J. Amer. Statist. Assoc., 109, 506, 686-699 (2014) · Zbl 1367.62223
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.