×

Sparsity considerations for dependent variables. (English) Zbl 1274.62462

Summary: The aim of this paper is to provide a comprehensive introduction for the study of \(\ell _{1}\)-penalized estimators in the context of dependent observations. We define a general \(\ell _{1}\)-penalized estimator for solving problems of stochastic optimization. This estimator turns out to be the LASSO [R. Tibshirani, J. R. Stat. Soc., Ser. B 58, No. 1, 267–288 (1996; Zbl 0850.62538)] in the regression estimation setting. Powerful theoretical guarantees on the statistical performances of the LASSO were provided in recent papers, however, they usually only deal with the iid case. Here, we study this estimator under various dependence assumptions.

MSC:

62J07 Ridge regression; shrinkage estimators (Lasso)
62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62J05 Linear regression; mixed models
62G07 Density estimation
62G08 Nonparametric regression and quantile regression

Citations:

Zbl 0850.62538

Software:

PDCO
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] H. Akaike. Information theory and an extension of the maximum likelihood principle. In B. N. Petrov and F. Csaki, editors, 2nd International Symposium on Information Theory , pages 267-281. Budapest: Akademia Kiado, 1973. · Zbl 0283.62006
[2] P. Alquier. Density estimation with quadratic loss, a confidence intervals method., ESAIM: P&S , 12:438-463, 2008. · Zbl 1182.62071 · doi:10.1051/ps:2007050
[3] A. Belloni and V. Chernozhukov., \ell 1 -penalized quantile regression in high-dimensional sparse models. Annals of Statistics , 32(11) :2011-2055, 2011. · Zbl 1209.62064 · doi:10.1214/10-AOS827
[4] A. Belloni and V. Chernozhukov. High dimensional sparse econometric models: An introduction. In P. Alquier, E. Gautier, and G. Stoltz, editors, Inverse Problems and High-Dimensional Estimation . Springer Lecture Notes in Statistics, 2011. · Zbl 1209.62064 · doi:10.1007/978-3-642-19989-9_3
[5] Y. Baraud, C. Giraud, and S. Huet. Gaussian model selection with an unknown variance., Annals of Statistics , 37(2):630-672, 2009. · Zbl 1162.62051 · doi:10.1214/07-AOS573
[6] K. Bartkiewicz, A. Jakubowskin, T. Mikosch, and O. Wintenberger. Infinite variances stable limits for sums of dependent random variables., Probability Theory and Related Fields , 2010.
[7] L. Birgé and P. Massart. Gaussian model selection., Journal of the European Mathematical Society , 3(3):203-268, 2001. · Zbl 1037.62001 · doi:10.1007/s100970100031
[8] P. J. Bickel, Y. Ritov, and A. Tsybakov. Simultaneous analysis of lasso and Dantzig selector., Annals of Statistics , 37(4) :1705-1732, 2009. · Zbl 1173.62022 · doi:10.1214/08-AOS620
[9] F. Bunea, A.B. Tsybakov, and M.H. Wegkamp. Aggregation for Gaussian regression., Annals of Statistics , 35 :1674-1697, 2007. · Zbl 1209.62065 · doi:10.1214/009053606000001587
[10] F. Bunea, A. Tsybakov, M. Wegkamp, and A. Barbu. SPADES and mixture models., Annals of Statistics , 38(4) :2525-2558, 2010. · Zbl 1198.62025 · doi:10.1214/09-AOS790
[11] F. Bunea, M. Wegkamp, and A. Tsybakov. Sparse density estimation with, \ell 1 penalties. Proceedings of 20th Annual Conference on Learning Theory (COLT 2007) - Springer , pages 530-543, 2007. · Zbl 1203.62053 · doi:10.1007/978-3-540-72927-3_38
[12] S. S. Chen, D. L. Donoho, and M. A. Saunders. Atomic decomposition by basis pursuit., SIAM Review , 43(1):129-159, 2001. · Zbl 0979.94010 · doi:10.1137/S003614450037906X
[13] S. S. Chen. Basis pursuit, 1995. PhD Thesis, Stanford, University.
[14] E. Candes and T. Tao. The Dantzig selector: statistical estimation when, p is much larger than n . Annals of Statistics , 35, 2007. · Zbl 1139.62019 · doi:10.1214/009053606000001523
[15] J. Dedecker, P. Doukhan, G. Lang, J. R. León R., S. Louhichi, and C. Prieur., Weak dependence: with examples and applications , volume 190 of Lecture Notes in Statistics . Springer, New York, 2007. · Zbl 1165.62001 · doi:10.1007/978-0-387-69952-3
[16] P. Doukhan and S. Louhichi. A new weak dependence condition and applications to moment inequalities., Stochastic Processes and their Applications , 84(2):313-342, 1999. · Zbl 0996.60020 · doi:10.1016/S0304-4149(99)00055-1
[17] P. Doukhan and M. H. Neumann. Probability and moment inequalities for sums of weakly dependent random variables, with applications., Stochastic Processes and their Applications , 117(7):878-903, 2007. · Zbl 1117.60018 · doi:10.1016/j.spa.2006.10.011
[18] P. Doukhan., Mixing , volume 85 of Lecture Notes in Statistics . Springer-Verlag, New York, 1994. Properties and examples.
[19] B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani. Least angle regression., Annals of Statistics , 32(2):407-499, 2004. · Zbl 1091.62054 · doi:10.1214/009053604000000067
[20] J. Friedman, T. Hastie, H. Höfling, and R. Tibshirani. Pathwise coordinate optimization., Annals of Applied Statist. , 1(2):302-332, 2007. · Zbl 1378.90064 · doi:10.1214/07-AOAS131
[21] M. Hebiri. Quelques questions de selection de variables autour de l’estimateur lasso, 2009. PhD Thesis, Université Paris 7 (in, english).
[22] W. Hoeffding. Probability inequalities for sums of bounded random variables., Journal of the American Statistical Association , 58:13-30, 1963. · Zbl 0127.10602 · doi:10.2307/2282952
[23] V. Koltchinskii. Sparsity in empirical risk minimization., Annales de l’Institut Henri Poincaré, Probabilités et Statistiques , 45(1):7-57, 2009. · Zbl 1168.62044 · doi:10.1214/07-AIHP146
[24] H. Leeb and B. M. Pötscher. Sparse estimators and the oracle property, or the return of hodges’ estimator. Cowles Foundation Discussion Papers 1500, Cowles Foundation, Yale University, 2005.
[25] F. Rapaport, E. Barillot, and J.-P. Vert. Classification of array-CGH data using fused SVM., Bioinformatics , 24(13) :1375, 1382, 2008.
[26] E. Rio., Théorie asymptotique pour des processus aléatoire faiblement dépendants . SMAI, Mathématiques et Applications 31, Springer, 2000.
[27] M. Rosenblatt. A central limit theorem and a strong mixing condition., Proc. Nat. Ac. Sc. U.S.A. , 42:43-47, 1956. · Zbl 0070.13804 · doi:10.1073/pnas.42.1.43
[28] M. Rosenblatt. Independence and dependence., Proceeding 4th. Berkeley Symp. Math. Stat. Prob. Berkeley University Press , pages 411-443, 1961.
[29] G. Schwarz. Estimating the dimension of a model., Annals of Statistics , 6:461-464, 1978. · Zbl 0379.62005 · doi:10.1214/aos/1176344136
[30] R. Tibshirani. Regression shrinkage and selection via the lasso., Journal of the Royal Statistical Society B , 58(1):267-288, 1996. · Zbl 0850.62538
[31] S. A. van de Geer and P. Bühlmann. On the conditions used to prove oracle results for the lasso., Electronic Journal of Statistics , 3 :1360-1392, 2009. · Zbl 1327.62425 · doi:10.1214/09-EJS506
[32] O. Wintenberger. Deviation inequalities for sums of weakly dependent time series., Electronic Communications in Probability , 15:489-503, 2010. · Zbl 1225.60034
[33] H. Zou and T. Hastie. Regularization and variable selection via the elastic net., Journal of the Royal Statistical Society B , 67(2):301-320, 2005. · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.