×

Regularization and variable selection via the elastic net. (English) Zbl 1069.62054

Summary: We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together. The elastic net is particularly useful when the number of predictors \((p)\) is much bigger than the number of observations \((n)\). By contrast, the lasso is not a very satisfactory variable selection method in the \(p\gg n\) case. An algorithm, called LARS-EN, is proposed for computing elastic net regularization paths efficiently, much like the algorithm LARS does for the lasso.

MSC:

62J05 Linear regression; mixed models
65C60 Computational problems in statistics (MSC2010)

Software:

ElemStatLearn; lars
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] DOI: 10.1214/aos/1032181158 · Zbl 0867.62055 · doi:10.1214/aos/1032181158
[2] DOI: 10.1016/j.jmva.2004.02.012 · Zbl 1047.62103 · doi:10.1016/j.jmva.2004.02.012
[3] Diaz-Uriarte R., Tech-nical Report (2003)
[4] Donoho D. L., J. R. Statist. Soc. 57 pp 301– (1995)
[5] DOI: 10.1214/009053604000000067 · Zbl 1091.62054 · doi:10.1214/009053604000000067
[6] DOI: 10.1198/016214501753382273 · Zbl 1073.62547 · doi:10.1198/016214501753382273
[7] Frank I., Technometrics 35 pp 109– (1993)
[8] Friedman J., J. Am. Statist. Ass. 84 pp 249– (1989)
[9] Friedman J., Ann. Statist. 32 pp 102– (2004)
[10] Fu W., J. Computnl Graph. Statist. 7 pp 397– (1998)
[11] Golub G., Matrix Computations (1983) · Zbl 0559.65011
[12] DOI: 10.1126/science.286.5439.531 · doi:10.1126/science.286.5439.531
[13] DOI: 10.1023/A:1012487302797 · Zbl 0998.68111 · doi:10.1023/A:1012487302797
[14] Hastie T., Genome Biol. 2 pp 0003.1– (2003)
[15] DOI: 10.1186/gb-2000-1-2-research0003 · doi:10.1186/gb-2000-1-2-research0003
[16] Hastie T., The Elements of Statistical Learning; Data Mining, Inference and Prediction (2001) · Zbl 0973.62007
[17] Hoerl A., Encyclopedia of Statistical Sciences pp 129– (1988)
[18] Rosset S., J. Mach. Learn. Res. 5 pp 941– (2004)
[19] DOI: 10.1089/106652703322756177 · doi:10.1089/106652703322756177
[20] Stamey T., J. Urol. 16 pp 1076– (1989)
[21] Tibshirani R., J. R. Statist. Soc. 58 pp 267– (1996)
[22] DOI: 10.1073/pnas.082099299 · doi:10.1073/pnas.082099299
[23] DOI: 10.1073/pnas.091062498 · Zbl 1012.92014 · doi:10.1073/pnas.091062498
[24] DOI: 10.1073/pnas.201162998 · doi:10.1073/pnas.201162998
[25] Zhang T., Ann. Statist. 32 pp 469– (2004)
[26] Zhu J., Biostatistics 5 pp 427– (2004)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.