Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors. (English) Zbl 1331.62359

Summary: Penalized regression is an attractive framework for variable selection problems. Often, variables possess a grouping structure, and the relevant selection problem is that of selecting groups, not individual variables. The group lasso has been proposed as a way of extending the ideas of the lasso to the problem of group selection. Nonconvex penalties such as SCAD and MCP have been proposed and shown to have several advantages over the lasso; these penalties may also be extended to the group selection problem, giving rise to group SCAD and group MCP methods. Here, we describe algorithms for fitting these models stably and efficiently. In addition, we present simulation results and real data examples comparing and contrasting the statistical properties of these methods.


62J12 Generalized linear models (logistic models)
62J07 Ridge regression; shrinkage estimators (Lasso)


S+WAVELETS; glmnet
Full Text: DOI arXiv Link


[1] Bakin, S.: Adaptive regression and model selection in data mining problems. Ph.D. thesis, Australian National University (1999) · Zbl 1137.62045
[2] Bertsekas, D.: Nonlinear Programming, 2nd edn. Athena Scientific, Nashua (1999) · Zbl 1015.90077
[3] Breheny, P.; Huang, J., Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection, Ann. Appl. Stat., 5, 232-253, (2011) · Zbl 1220.62095
[4] Chiang, A.; Beck, J.; Yen, H.; Tayeh, M.; Scheetz, T.; Swiderski, R.; Nishimura, D.; Braun, T.; Kim, K.; Huang, J.; etal., Homozygosity mapping with SNP arrays identifies TRIM32, an E3 ubiquitin ligase, as a bardet-biedl syndrome gene (BBS11), Proc. Natl. Acad. Sci., 103, 6287-6292, (2006)
[5] Donoho, D.; Johnstone, J., Ideal spatial adaptation by wavelet shrinkage, Biometrika, 81, 425-455, (1994) · Zbl 0815.62019
[6] Fan, J.; Li, R., Variable selection via nonconcave penalized likelihood and its oracle properties, J. Am. Stat. Assoc., 96, 1348-1360, (2001) · Zbl 1073.62547
[7] Foygel, R., Drton, M.: Exact block-wise optimization in group lasso and sparse group lasso for linear regression (2010). Arxiv preprint arXiv:1010.3320 · Zbl 1142.62027
[8] Friedman, J.; Hastie, T.; Höfling, H.; Tibshirani, R., Pathwise coordinate optimization, Ann. Appl. Stat., 1, 302-332, (2007) · Zbl 1378.90064
[9] Friedman, J., Hastie, T., Tibshirani, R.: A note on the group lasso and a sparse group lasso (2010a). Arxiv preprint arXiv:1001.0736 · Zbl 1141.62030
[10] Friedman, J.H.; Hastie, T.; Tibshirani, R., Regularization paths for generalized linear models via coordinate descent, J. Stat. Softw., 33, 1-22, (2010)
[11] Fu, W., Penalized regressions: the bridge versus the lasso, J. Comput. Graph. Stat., 7, 397-416, (1998)
[12] Gao, H.; Bruce, A., Waveshrink with firm shrinkage, Stat. Sin., 7, 855-874, (1997) · Zbl 1067.62529
[13] Huang, J.; Breheny, P.; Ma, S., A selective review of group selection in high-dimensional models, Stat. Sci., 27, 481-499, (2012) · Zbl 1331.62347
[14] Hunter, D.; Lange, K., A tutorial on MM algorithms, Am. Stat., 58, 30-38, (2004)
[15] Krishnapuram, B.; Carin, L.; Figueiredo, M.; Hartemink, A., Sparse multinomial logistic regression: fast algorithms and generalization bounds, IEEE Trans. Pattern Anal. Mach. Intell., 27, 957-968, (2005)
[16] Lange, K.; Hunter, D.; Yang, I., Optimization transfer using surrogate objective functions, J. Comput. Graph. Stat., 9, 1-20, (2000)
[17] Meier, L.; Geer, S.; Buhlmann, P., The group lasso for logistic regression, J. R. Stat. Soc. B, 70, 53, (2008) · Zbl 1400.62276
[18] Puig, A.; Wiesel, A.; Fleury, G.; Hero, A., Multidimensional shrinkage-thresholding operator and group lasso penalties, IEEE Signal Process. Lett., 18, 363-366, (2011)
[19] Ravikumar, P.; Lafferty, J.; Liu, H.; Wasserman, L., Sparse additive models, J. R. Stat. Soc. B, 71, 1009-1030, (2009) · Zbl 1411.62107
[20] Scheetz, T.; Kim, K.; Swiderski, R.; Philp, A.; Braun, T.; Knudtson, K.; Dorrance, A.; DiBona, G.; Huang, J.; Casavant, T.; etal., Regulation of gene expression in the Mammalian eye and its relevance to eye disease, Proc. Natl. Acad. Sci., 103, 14429-14434, (2006)
[21] She, Y., An iterative algorithm for Fitting nonconvex penalized generalized linear models with grouped predictors, Comput. Stat. Data Anal., 56, 2976-2990, (2012) · Zbl 1255.62209
[22] Simon, N.; Tibshirani, R., Standardization and the group lasso penalty, Stat. Sin., 22, 983-1001, (2011) · Zbl 1257.62080
[23] Tibshirani, R., Regression shrinkage and selection via the lasso, J. R. Stat. Soc. B, 58, 267-288, (1996) · Zbl 0850.62538
[24] Tseng, P., Convergence of a block coordinate descent method for nondifferentiable minimization, J. Optim. Theory Appl., 109, 475-494, (2001) · Zbl 1006.65062
[25] Wang, L.; Chen, G.; Li, H., Group SCAD regression analysis for microarray time course gene expression data, Bioinformatics, 23, 1486, (2007)
[26] Wang, L.; Li, H.; Huang, J.Z., Variable selection in nonparametric varying-coefficient models for analysis of repeated measurements, J. Am. Stat. Assoc., 103, 1556-1569, (2008) · Zbl 1286.62034
[27] Wu, T.; Lange, K., Coordinate descent algorithms for lasso penalized regression, Ann. Appl. Stat., 2, 224-244, (2008) · Zbl 1137.62045
[28] Yuan, M.; Lin, Y., Model selection and estimation in regression with grouped variables, J. R. Stat. Soc. B, 68, 49-67, (2006) · Zbl 1141.62030
[29] Zhang, C., Nearly unbiased variable selection under minimax concave penalty, Ann. Stat., 38, 894-942, (2010) · Zbl 1183.62120
[30] Zou, H.; Li, R., One-step sparse estimates in nonconcave penalized likelihood models, Ann. Stat., 36, 1509-1533, (2008) · Zbl 1142.62027
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.