Regularization and variable selection via the elastic net. (English) Zbl 1069.62054

Zou, Hui; Hastie, Trevor

J. R. Stat. Soc., Ser. B, Stat. Methodol. 67, No. 2, 301-320 (2005).

Summary: We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together. The elastic net is particularly useful when the number of predictors \((p)\) is much bigger than the number of observations \((n)\). By contrast, the lasso is not a very satisfactory variable selection method in the \(p\gg n\) case. An algorithm, called LARS-EN, is proposed for computing elastic net regularization paths efficiently, much like the algorithm LARS does for the lasso.

MSC:

62J05	Linear regression; mixed models
65C60	Computational problems in statistics (MSC2010)

Keywords:

grouping effect; LARS algorithm; lasso; penalization; \(p\gg n\) problem; variable selection; prostate cancer

Software:

ElemStatLearn; lars

References:

[1]	DOI: 10.1214/aos/1032181158 · Zbl 0867.62055
[2]	DOI: 10.1016/j.jmva.2004.02.012 · Zbl 1047.62103
[3]	Diaz-Uriarte R., Tech-nical Report (2003)
[4]	Donoho D. L., J. R. Statist. Soc. 57 pp 301– (1995)
[5]	DOI: 10.1214/009053604000000067 · Zbl 1091.62054
[6]	DOI: 10.1198/016214501753382273 · Zbl 1073.62547
[7]	Frank I., Technometrics 35 pp 109– (1993)
[8]	Friedman J., J. Am. Statist. Ass. 84 pp 249– (1989)
[9]	Friedman J., Ann. Statist. 32 pp 102– (2004)
[10]	Fu W., J. Computnl Graph. Statist. 7 pp 397– (1998)
[11]	Golub G., Matrix Computations (1983) · Zbl 0559.65011
[12]	DOI: 10.1126/science.286.5439.531
[13]	DOI: 10.1023/A:1012487302797 · Zbl 0998.68111
[14]	Hastie T., Genome Biol. 2 pp 0003.1– (2003)
[15]	DOI: 10.1186/gb-2000-1-2-research0003
[16]	Hastie T., The Elements of Statistical Learning; Data Mining, Inference and Prediction (2001) · Zbl 0973.62007
[17]	Hoerl A., Encyclopedia of Statistical Sciences pp 129– (1988)
[18]	Rosset S., J. Mach. Learn. Res. 5 pp 941– (2004)
[19]	DOI: 10.1089/106652703322756177
[20]	Stamey T., J. Urol. 16 pp 1076– (1989)
[21]	Tibshirani R., J. R. Statist. Soc. 58 pp 267– (1996)
[22]	DOI: 10.1073/pnas.082099299
[23]	DOI: 10.1073/pnas.091062498 · Zbl 1012.92014
[24]	DOI: 10.1073/pnas.201162998
[25]	Zhang T., Ann. Statist. 32 pp 469– (2004)
[26]	Zhu J., Biostatistics 5 pp 427– (2004)