×

zbMATH — the first resource for mathematics

Regularization and variable selection via the elastic net. (English) Zbl 1069.62054
Summary: We propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together. The elastic net is particularly useful when the number of predictors \((p)\) is much bigger than the number of observations \((n)\). By contrast, the lasso is not a very satisfactory variable selection method in the \(p\gg n\) case. An algorithm, called LARS-EN, is proposed for computing elastic net regularization paths efficiently, much like the algorithm LARS does for the lasso.

MSC:
62J05 Linear regression; mixed models
65C60 Computational problems in statistics (MSC2010)
Software:
ElemStatLearn; lars
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] DOI: 10.1214/aos/1032181158 · Zbl 0867.62055 · doi:10.1214/aos/1032181158
[2] DOI: 10.1016/j.jmva.2004.02.012 · Zbl 1047.62103 · doi:10.1016/j.jmva.2004.02.012
[3] Diaz-Uriarte R., Tech-nical Report (2003)
[4] Donoho D. L., J. R. Statist. Soc. 57 pp 301– (1995)
[5] DOI: 10.1214/009053604000000067 · Zbl 1091.62054 · doi:10.1214/009053604000000067
[6] DOI: 10.1198/016214501753382273 · Zbl 1073.62547 · doi:10.1198/016214501753382273
[7] Frank I., Technometrics 35 pp 109– (1993)
[8] Friedman J., J. Am. Statist. Ass. 84 pp 249– (1989)
[9] Friedman J., Ann. Statist. 32 pp 102– (2004)
[10] Fu W., J. Computnl Graph. Statist. 7 pp 397– (1998)
[11] Golub G., Matrix Computations (1983) · Zbl 0559.65011
[12] DOI: 10.1126/science.286.5439.531 · doi:10.1126/science.286.5439.531
[13] DOI: 10.1023/A:1012487302797 · Zbl 0998.68111 · doi:10.1023/A:1012487302797
[14] Hastie T., Genome Biol. 2 pp 0003.1– (2003)
[15] DOI: 10.1186/gb-2000-1-2-research0003 · doi:10.1186/gb-2000-1-2-research0003
[16] Hastie T., The Elements of Statistical Learning; Data Mining, Inference and Prediction (2001) · Zbl 0973.62007
[17] Hoerl A., Encyclopedia of Statistical Sciences pp 129– (1988)
[18] Rosset S., J. Mach. Learn. Res. 5 pp 941– (2004)
[19] DOI: 10.1089/106652703322756177 · doi:10.1089/106652703322756177
[20] Stamey T., J. Urol. 16 pp 1076– (1989)
[21] Tibshirani R., J. R. Statist. Soc. 58 pp 267– (1996)
[22] DOI: 10.1073/pnas.082099299 · doi:10.1073/pnas.082099299
[23] DOI: 10.1073/pnas.091062498 · Zbl 1012.92014 · doi:10.1073/pnas.091062498
[24] DOI: 10.1073/pnas.201162998 · doi:10.1073/pnas.201162998
[25] Zhang T., Ann. Statist. 32 pp 469– (2004)
[26] Zhu J., Biostatistics 5 pp 427– (2004)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.