Better subset regression using the nonnegative garrote. (English) Zbl 0862.62059

Breiman, Leo

Technometrics 37, No. 4, 373-384 (1995).

Summary: A new method, called the nonnegative (nn) garrote, is proposed for doing subset regression. It both shrinks and zeros coefficients. In tests on real and simulated data, it produces lower prediction errors than ordinary subset selection. It is also compared to ridge regression. If the regression equations generated by a procedure do not change drastically with small changes in the data, the procedure is called stable. Subset selection is unstable, ridge is very stable, and the nn-garrote is intermediate. Simulation results illustrate the effects of instability on prediction error.

MSC:

62J05	Linear regression; mixed models
62J07	Ridge regression; shrinkage estimators (Lasso)
65C05	Monte Carlo methods

Keywords:

nonnegative garrote; simulation results; little bootstrap; model error; stability; subset regression; prediction error; subset selection; ridge regression