×

zbMATH — the first resource for mathematics

Non-asymptotic oracle inequalities for the Lasso and group Lasso in high dimensional logistic model. (English) Zbl 1353.62054
Summary: We consider the problem of estimating a function \(f_{0}\) in logistic regression model. We propose to estimate this function \(f_{0}\) by a sparse approximation build as a linear combination of elements of a given dictionary of \(p\) functions. This sparse approximation is selected by the Lasso or Group Lasso procedure. In this context, we state non asymptotic oracle inequalities for Lasso and Group Lasso under restricted eigenvalue assumption as introduced in [P. J. Bickel et al., Ann. Stat. 37, No. 4, 1705–1732 (2009; Zbl 1173.62022)].

MSC:
62H12 Estimation in multivariate analysis
62J12 Generalized linear models (logistic models)
62J07 Ridge regression; shrinkage estimators (Lasso)
62G20 Asymptotic properties of nonparametric inference
Software:
glmnet; hgam
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] H. Akaike, Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory (Tsahkadsor, 1971). Akadémiai Kiadó, Budapest (1973) 267-281. · Zbl 0283.62006
[2] F. Bach, Self-concordant analysis for logistic regression. Electron. J. Statist.4 (2010) 384-414. · Zbl 1329.62324 · doi:10.1214/09-EJS521
[3] P.L. Bartlett, S. Mendelson and J. Neeman, 1-regularized linear regression: persistence and oracle inequalities. Probab. Theory Relat. Fields154 (2012) 193-224. · Zbl 1395.62207 · doi:10.1007/s00440-011-0367-2
[4] P.J. Bickel, Y. Ritov and A.B. Tsybakov, Simultaneous analysis of Lasso and Dantzig selector. Ann. Statist.37 (2009) 1705-1732. · Zbl 1173.62022 · doi:10.1214/08-AOS620
[5] S. Boucheron, G. Lugosi and O. Bousquet, Concentration inequalities. Adv. Lect. Machine Learn. (2004) 208-240. · Zbl 1120.68427
[6] F. Bunea, A.B. Tsybakov and M.H. Wegkamp, Aggregation and sparsity via l_1 penalized least squares. In Learning theory, vol. 4005 of Lect. Notes Comput. Sci. Springer, Berlin (2006) 379-391. · Zbl 1143.62319
[7] F. Bunea, A.B. Tsybakov and M.H. Wegkamp, Aggregation for Gaussian regression. Ann. Statist.35 (2007) 1674-1697. · Zbl 1209.62065 · doi:10.1214/009053606000001587
[8] F. Bunea, A. Tsybakov and M. Wegkamp, Sparsity oracle inequalities for the Lasso. Electron. J. Statist.1 (2007) 169-194. · Zbl 1146.62028 · doi:10.1214/07-EJS008
[9] C. Chesneau and M. Hebiri, Some theoretical results on the grouped variables lasso. Math. Methods Statist.17 (2008) 317-326. · Zbl 1282.62159 · doi:10.3103/S1066530708040030
[10] J. Friedman, T. Hastie and R. Tibshirani, Regularization paths for generalized linear models via coordinate descent. J. Statist. Software33 (2010) 1. · doi:10.18637/jss.v033.i01
[11] M. Garcia-Magariños, A. Antoniadis, R. Cao and W. González-Manteiga, Lasso logistic regression, GSoft and the cyclic coordinate descent algorithm: application to gene expression data. Stat. Appl. Genet. Mol. Biol.9 (2010) 30. · Zbl 1304.92087
[12] T. Hastie, Non-parametric logistic regression. SLAC PUB-3160 (1983).
[13] J. Huang, S. Ma and CH Zhang, The iterated lasso for high-dimensional logistic regression. Technical Report 392 (2008).
[14] J. Huang, J.L. Horowitz and F. Wei, Variable selection in nonparametric additive models. Ann. Statist.38 (2010) 2282. · Zbl 1202.62051 · doi:10.1214/09-AOS781
[15] G.M James, P. Radchenko and J. Lv, Dasso: connections between the dantzig selector and lasso. J. Roy. Statist. Soc. Ser. B71 (2009) 127-142. · Zbl 1231.62129 · doi:10.1111/j.1467-9868.2008.00668.x
[16] K. Knight and W. Fu, Asymptotics for lasso-type estimators. Ann. Statist.28 (2000) 1356-1378. · Zbl 1105.62357 · doi:10.1214/aos/1015957397
[17] K. Lounici, M. Pontil, A.B. Tsybakov and S. Van De Geer, Taking advantage of sparsity in multi-task learning. In COLT’09 (2009).
[18] K. Lounici, M. Pontil, S. van de Geer and A.B. Tsybakov, Oracle inequalities and optimal inference under group sparsity. Ann. Statist.39 (2011) 2164-2204. · Zbl 1306.62156 · doi:10.1214/11-AOS896
[19] P. Massart, Concentration inequalities and model selection. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6-23, 2003. With a foreword by Jean Picard. Vol. 1896 of Lect. Notes Math. Springer, Berlin (2007).
[20] P. Massart and C. Meynet, The Lasso as an _1-ball model selection procedure. Electron. J. Statist.5 (2011) 669-687. · Zbl 1274.62468 · doi:10.1214/11-EJS623
[21] J. McAuley, J. Ming, D. Stewart and P. Hanna, Subband correlation and robust speech recognition. IEEE Trans. Speech Audio Process.13 (2005) 956-964. · doi:10.1109/TSA.2005.851952
[22] L. Meier, S. van de Geer and P. Bühlmann, The group Lasso for logistic regression. J. Roy. Statist. Soc. Ser. B70 (2008) 53-71. · Zbl 1400.62276 · doi:10.1111/j.1467-9868.2007.00627.x
[23] L. Meier, S. van de Geer and P. Bühlmann, High-dimensional additive modeling. Ann. Statist.37 (2009) 3779-3821. · Zbl 1360.62186 · doi:10.1214/09-AOS692
[24] N. Meinshausen and P. Bühlmann, High-dimensional graphs and variable selection with the lasso. Ann. Statist.34 (2006) 1436-1462. · Zbl 1113.62082 · doi:10.1214/009053606000000281
[25] N. Meinshausen and B. Yu, Lasso-type recovery of sparse representations for high-dimensional data. Ann. Statist.37 (2009) 246-270. · Zbl 1155.62050 · doi:10.1214/07-AOS582
[26] Y. Nardi and A. Rinaldo, On the asymptotic properties of the group lasso estimator for linear models. Electron. J. Statist.2 (2008) 605-633. · Zbl 1320.62167 · doi:10.1214/08-EJS200
[27] S.N. Negahban, P. Ravikumar, M.J. Wainwright and B. Yu, A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. Statist. Sci.27 (2012) 538-557. · Zbl 1331.62350 · doi:10.1214/12-STS400
[28] Y. Nesterov and A. Nemirovskii, Interior-point polynomial algorithms in convex programming. Vol. 13 of SIAM Studies in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (1994). · Zbl 0824.90112
[29] M.R. Osborne, B. Presnell and B.A. Turlach, A new approach to variable selection in least squares problems. IMA J. Numer. Anal.20 (2000) 389-403. · Zbl 0962.65036 · doi:10.1093/imanum/20.3.389
[30] M.Y. Park and T. Hastie, L_1-regularization path algorithm for generalized linear models. J. Roy. Statist. Soc. Ser. B69 (2007) 659-677. · doi:10.1111/j.1467-9868.2007.00607.x
[31] B. Tarigan and S.A. van de Geer, Classifiers of support vector machine type with l_1 complexity regularization. Bernoulli12 (2006) 1045-1076. · Zbl 1118.62067 · doi:10.3150/bj/1165269150
[32] R. Tibshirani, Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B58 (1996) 267-288. · Zbl 0850.62538
[33] P. Ravikumar, J. Lafferty, H. Liu and L. Wasserman, Sparse additive models. J. Roy. Statist. Soc. Ser. B71 (2009) 1009-1030. · doi:10.1111/j.1467-9868.2009.00718.x
[34] G. Schwarz, Estimating the dimension of a model. Ann. Statist.6 (1978) 461-464. · Zbl 0379.62005 · doi:10.1214/aos/1176344136
[35] S.A. van de Geer, High-dimensional generalized linear models and the lasso. Ann. Statist.36 (2008) 614-645. · Zbl 1138.62323 · doi:10.1214/009053607000000929
[36] S.A. van de Geer and P. Bühlmann, On the conditions used to prove oracle results for the Lasso. Electron. J. Statist.3 (2009) 1360-1392. · Zbl 1327.62425 · doi:10.1214/09-EJS506
[37] T.T. Wu, Y.F. Chen, T. Hastie, E. Sobel and K. Lange, Genome-wide association analysis by lasso penalized logistic regression. Bioinform.25 (2009) 714-721. · Zbl 05743814 · doi:10.1093/bioinformatics/btp041
[38] M. Yuan and Y. Lin, Model selection and estimation in regression with grouped variables. J. Roy. Statist. Soc. Ser. B68 (2006) 49-67. · Zbl 1141.62030 · doi:10.1111/j.1467-9868.2005.00532.x
[39] C.-H. Zhang and J. Huang, The sparsity and bias of the LASSO selection in high-dimensional linear regression. Ann. Statist.36 (2008) 1567-1594. · Zbl 1142.62044 · doi:10.1214/07-AOS520
[40] P. Zhao and B. Yu, On model selection consistency of Lasso. J. Mach. Learn. Res.7 (2006) 2541-2563. · Zbl 1222.62008
[41] H. Zou, The adaptive lasso and its oracle properties. J. Am. Statist. Assoc.101 (2006) 1418-1429. · Zbl 1171.62326 · doi:10.1198/016214506000000735
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.