×

zbMATH — the first resource for mathematics

A unified penalized method for sparse additive quantile models: an RKHS approach. (English) Zbl 1447.62044
The paper puts forward an additive quantile regression model for high dimension and sparsity versus small sample size data with a novel sparsity-smoothness penalty under the reproducing kernel Hilbert space. The introduced model is optimized through a hybridization of the majorize minimization and the proximal gradient approaches. Additionally, oracle inequalities are set under weak conditions. The theoretical flow is accompanied by simulations and real-world test cases.

MSC:
62G08 Nonparametric regression and quantile regression
62J07 Ridge regression; shrinkage estimators (Lasso)
62-08 Computational methods for problems pertaining to statistics
46E22 Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces)
Software:
hgam
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Bartlett, P. L., Bousquet, O., Mendelson, S. (2005). Local Rademacher complexities. Annals of Statistics, 33, 1497-1537. · Zbl 1083.62034
[2] Beck, A., Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, \(2\), 183-202. · Zbl 1175.94009
[3] Belloni, A., Chernozhukov, V. (2011). \(ℓ _1\) penalized quantile regression in high-dimensional sparse models. Annals of Statistics, 39, 83-130. · Zbl 1209.62064
[4] Combettes, P., Wajs, V. (2005). Signal recovery by proximal forward-backward splitting. Multiscale Modeling and Simulation, \(4\), 1168-1200. · Zbl 1179.94031
[5] Donoho, D. L., Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association, 90, 1200-1224. · Zbl 0869.62024
[6] Fan, J., Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96, 1348-1360. · Zbl 1073.62547
[7] Hastie, T., Tibshirani, R. (1990). Monographs on Statistics and Applied Probability, Generalized Additive Models (1st ed.), London: Chapman and Hall. · Zbl 0747.62061
[8] He, X. M., Wang, L., Hong, H. G. (2013). Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data. Annals of Statistics, 41, 324-369. · Zbl 1295.62053
[9] Hunter, D. R., Lange, K. (2000). Quantile regression via an MM algorithm. Journal of Computational and Graphical Statistics, 11, 60-77. · Zbl 0118.10502
[10] Jaakkola, T., Diekhans, M., Haussler, D. (1999). Using the Fisher kernel method to detect remote protein homologies. In Proceedings of Seventh International Conference on Intelligent Systems for Molecular Biology, 149-158. · Zbl 1432.62090
[11] Kato, K. (2016). Group Lasso for high dimensional sparse quantile regression models. Manuscript.
[12] Koenker, R, Additive models for quantile regression: model selection and confidence bandaids, Brazilian Journal of Probability and Statistics, 25, 239-262, (2011) · Zbl 1236.62031
[13] Koenker, R., Basset, G. (1978). Regression quantiles. Econometrica, 46, 33-50. · Zbl 0373.62038
[14] Koltchinskii, V., Yuan, M. (2008). Sparse recovery in large ensembles of kerenl machines. In: 21st Annual Conference on Learning Theory, Helsinki, 229-238.
[15] Koltchinskii, V., Yuan, M. (2010). Sparsity in multiple kernel learning. Annals of Statistics, 38, 3660-3695. · Zbl 1204.62086
[16] Li, Y., Zhu, J. (2008). \(l^1\)-norm quantile regressions. Journal of Computational and Graphical Statistics, 17, 163-185. · Zbl 0118.10502
[17] Li, Y., Liu, Y., Zhu, J. (2007). Quantile regression in reproducing kernel Hilbert spaces. Journal of the American Statistical Association, 102, 255-268. · Zbl 1284.62405
[18] Lian, H, Estimation of additive quantile regression models by two-fold penalty, Journal of Business and Economic Statistics, 30, 337-350, (2012)
[19] Lv, S. G., Lin, H. Z., Lian, H., Huang, J. (2016). Oracle inequalities for sparse additive quantile regression models in reproducing kernel Hilbert space. Manuscript. · Zbl 06870279
[20] Meier, L., Van der Geer, S., Bühlmann, P. (2009). High-dimensional additive modeling. Annals of Statistics, 37, 3779-3821. · Zbl 1360.62186
[21] Mernshausen, N., Yu, B. (2009). Lasso-type recovery of sparse representations for high-dimensional data. Annals of Statistics, 37, 246-270. · Zbl 1155.62050
[22] Moreau, JJ, Fonctions convexes duales et points proximaux dans un espace hilbertien, Reports of the Paris Academy of Sciences, Series A, 255, 2897-2899, (1962) · Zbl 0118.10502
[23] Mosci, S., Rosasco, L., Santoro, M., Verri, A., Villa, S. (2010). Solving structured sparsity regularization with proximal methods. Machine Learning and Knowledge Discovery in Databases, 6322, 418-433. · Zbl 1317.68183
[24] Negahban, S., Ravikumar, P., Wainwright, M. J., Yu, B. (2012). A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. Statistical Science, 27, 538-557. · Zbl 1331.62350
[25] Pearce, N. D., Wand, M. P. (2006). Penalized splines and reproducing kernel methods. The American Statistician, 60, 233-240.
[26] Raskutti, G., Wainwright, M., Yu, B. (2012). Minimax-optimal rates for sparse additive models over kernel classes via convex programming. Journal of Machine Learning Research, 13, 389-427. · Zbl 1283.62071
[27] Ravikumar, P., Liu, H., Lafferty, J., Wasserman, L. (2009). SpAM: Sparse additive models. Journal of the Royal Statistical Society: Series B, 71, 1009-1030. · Zbl 1411.62107
[28] Rosasco, L., Villa, S., Mosci, S., Santoro, M., Verri, A. (2013). Nonparametric sparsity and regularization. Journal of Machine Learning Research, 14, 1665-1714. · Zbl 1317.68183
[29] Steinwart, I., Christmann, A. (2011). Estimating conditional quantiles with the help of pinball loss. Bernoulli, 17, 211-225. · Zbl 1284.62235
[30] Tseng, P, Approximation accuracy, gradient methods, and error bound for structured convex optimization, Mathematical Programming, 125, 263-295, (2010) · Zbl 1207.65084
[31] Van der Geer, S. (2000). Empirical Processes in M-estimation. Cambridge: Cambridge University Press.
[32] Van der Geer, S. (2008). High-dimensional generalized linear models and the Lasso. Annals of Statistics, 36, 614-645. · Zbl 1138.62323
[33] Wahba, G. (1999). Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV. Advances in kernel methods: support vector learning, pp. 69-88.
[34] Wang, L., Wu, Y. C., Li, R. Z. (2012). Quantile regression for analyzing heterogeneity in ultra-high dimension. Journal of the American Statistical Association, 107, 214-222. · Zbl 1328.62468
[35] Xue, L, Consistent variable selection in additive models, Statistical Science, 19, 1281-1296, (2009) · Zbl 1166.62024
[36] Yafeh, Y., Yosha, O. (2003). Large Shareholders and banks: Who monitors and how? The Economic Journal, 113, 128-146.
[37] Yuan, M, GACV for quantile smoothing splines, Computational Statistics and Data Analysis, 5, 813-829, (2006) · Zbl 1432.62090
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.