×

zbMATH — the first resource for mathematics

Degrees of freedom in lasso problems. (English) Zbl 1274.62469
Summary: We derive the degrees of freedom of the Lasso fit, placing no assumptions on the predictor matrix \(X\). Like the well-known result of H. Zou, T. Hastie and R. Tibshirani [Ann. Stat. 35, No. 5, 2173–2192 (2007; Zbl 1126.62061)], which gives the degrees of freedom of the Lasso fit when \(X\) has full column rank, we express our result in terms of the active set of a Lasso solution. We extend this result to cover the degrees of freedom of the generalized Lasso fit for an arbitrary predictor matrix \(X\) (and an arbitrary penalty matrix \(D\)). Though our focus is degrees of freedom, we establish some intermediate results on the Lasso and generalized Lasso that may be interesting on their own.

MSC:
62J07 Ridge regression; shrinkage estimators (Lasso)
90C46 Optimality conditions and duality in mathematical programming
Software:
PDCO
PDF BibTeX XML Cite
Full Text: DOI Euclid arXiv
References:
[1] Chen, S. S., Donoho, D. L. and Saunders, M. A. (1998). Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20 33-61. · Zbl 0919.94002
[2] Dossal, C., Kachour, M., Fadili, J., Peyre, G. and Chesneau, C. (2011). The degrees of freedom of the lasso for general design matrix. Available at . · Zbl 1433.62193
[3] Efron, B. (1986). How biased is the apparent error rate of a prediction rule? J. Amer. Statist. Assoc. 81 461-470. · Zbl 0621.62073
[4] Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression (with discussion, and a rejoinder by the authors). Ann. Statist. 32 407-499. · Zbl 1091.62054
[5] Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. · Zbl 1073.62547
[6] Grünbaum, B. (2003). Convex Polytopes , 2nd ed. Graduate Texts in Mathematics 221 . Springer, New York. · Zbl 1033.52001
[7] Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Monographs on Statistics and Applied Probability 43 . Chapman & Hall, London. · Zbl 0747.62061
[8] Loubes, J. M. and Massart, P. (2004). Dicussion to “Least angle regression.” Ann. Statist. 32 460-465. · Zbl 1091.62054
[9] Mallows, C. (1973). Some comments on \(C_p\). Technometrics 15 661-675. · Zbl 0269.62061
[10] Meyer, M. and Woodroofe, M. (2000). On the degrees of freedom in shape-restricted regression. Ann. Statist. 28 1083-1104. · Zbl 1105.62340
[11] Osborne, M. R., Presnell, B. and Turlach, B. A. (2000). On the LASSO and its dual. J. Comput. Graph. Statist. 9 319-337.
[12] Rosset, S., Zhu, J. and Hastie, T. (2004). Boosting as a regularized path to a maximum margin classifier. J. Mach. Learn. Res. 5 941-973. · Zbl 1222.68290
[13] Schneider, R. (1993). Convex Bodies : The Brunn-Minkowski Theory. Encyclopedia of Mathematics and Its Applications 44 . Cambridge Univ. Press, Cambridge. · Zbl 0798.52001
[14] Stein, C. M. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135-1151. · Zbl 0476.62035
[15] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267-288. · Zbl 0850.62538
[16] Tibshirani, R. J. (2011). The solution path of the generalized lasso, Ph.D. thesis, Dept. Statistics, Stanford Univ. · Zbl 1234.62107
[17] Tibshirani, R. J. and Taylor, J. (2011). The solution path of the generalized lasso. Ann. Statist. 39 1335-1371. · Zbl 1234.62107
[18] Vaiter, S., Peyre, G., Dossal, C. and Fadili, J. (2011). Robust sparse analysis regularization. Available at . · Zbl 1229.62157
[19] Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 301-320. · Zbl 1069.62054
[20] Zou, H., Hastie, T. and Tibshirani, R. (2007). On the “degrees of freedom” of the lasso. Ann. Statist. 35 2173-2192. · Zbl 1126.62061
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.