×

On proximal gradient method for the convex problems regularized with the group reproducing kernel norm. (English) Zbl 1318.90055

Summary: We consider a class of nonsmooth convex optimization problems where the objective function is the composition of a strongly convex differentiable function with a linear mapping, regularized by the group reproducing kernel norm. This class of problems arise naturally from applications in group Lasso, which is a popular technique for variable selection. An effective approach to solve such problems is by the proximal gradient method. In this paper we derive and study theoretically the efficient algorithms for the class of the convex problems, analyze the convergence of the algorithm and its subalgorithm.

MSC:

90C25 Convex programming
90C52 Methods of reduced gradient type

Software:

UNLocBoX; SLEP
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Bach, F.: Consistency of the group Lasso and multiple kernel learning. J. Mach. Learn. Res. 9, 1179-1225 (2008) · Zbl 1225.68147
[2] Bakin, S.: Adaptive Regression and Model Selection in Data Mining Problems. PhD Thesis. Australian National University, Canberra (1999) · Zbl 1171.62326
[3] Boikanyo, O.A., Morosanu, G.: Four parameter proximal point algorithms. Nonlinear Anal. 74, 544-C555 (2011) · Zbl 1298.47075 · doi:10.1016/j.na.2010.09.008
[4] Boikanyo, O.A., Morosanu, G.: Inexact Halpern-type proximal point algorithm. J. Glob. Optim. 51, 11C26 (2011). doi: 10.1007/s10898-010-9616-7 · Zbl 1295.47073
[5] Cartis, C., Gould, N.I.M., Ph. Toint, L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. Ser. A 127, 245-295 (2011) · Zbl 1229.90192
[6] Combettes, P.L., Pesquet J.C.: Proximal Splitting Methods in Signal Processing. arXiv:0912.3522v4 [math.OC] 18 May (2010) · Zbl 1242.90160
[7] Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4, 1168-1200 (2005) · Zbl 1179.94031 · doi:10.1137/050626090
[8] Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293-318 (1992) · Zbl 0765.90073 · doi:10.1007/BF01581204
[9] Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96(456), 1348-1359 (2001) · Zbl 1073.62547 · doi:10.1198/016214501753382273
[10] Friedman, J., Hastie, T., Tibshirani, R.: A Note on the Group Lasso and a Sparse Group Lasso. arXiv:1001.0736v1 [math.ST] 5 Jan (2010) · Zbl 1391.94442
[11] Kelley, C.T.: Iterative Methods for Linear and Nonlinear Equations. Society for Industrial and Applied Mathematics, Philadelphia (1995) · Zbl 0832.65046 · doi:10.1137/1.9781611970944
[12] Kim, D., Sra, S., Dhillon, I.: A scalable trust-region algorithm with application to mixednorm regression. In: International Conference on Machine Learning (ICML), p. 1 (2010)
[13] Liu, J., Ji, S., Ye, J.: SLEP: Sparse Learning with Efficient Projections. Arizona State University (2009)
[14] Luo, Z.Q., Tseng, P.: On the linear convergence of descent methods for convex essentially smooth minimization. SIAM J. Control Optim. 30(2), 408-425 (1992) · Zbl 0756.90084
[15] Ma, S., Song, X., Huang, J.: Supervised group Lasso with applications to microarray data analysis. BMC Bioinform. 8(1), 60 (2007) · doi:10.1186/1471-2105-8-60
[16] Meier, L., Van De Geer, S., Buhlmann, P.: The group Lasso for logistic regression. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 70(1), 53-71 (2008) · Zbl 1400.62276 · doi:10.1111/j.1467-9868.2007.00627.x
[17] Nesterov, Y.: Introductory Lectures on Convex Optimization. Kluwer, Boston (2004) · Zbl 1086.90045 · doi:10.1007/978-1-4419-8853-9
[18] Paul, H.Calamai, Jorge, J.Mor: Projected gradient methods for linearly constrained problems. Math. Progrom. 39, 93-116 (1987) · Zbl 0634.90064 · doi:10.1007/BF02592073
[19] Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970) · Zbl 0193.18401
[20] Rockafellar, R.T., Wets, R.J.B.: Variational Analysis. Springer, New York (1998) · Zbl 0888.49001 · doi:10.1007/978-3-642-02431-3
[21] Roth, V., Fischer, B.: The group-Lasso for generalized linear models: uniqueness of solutions and efficient algorithms. In: Proceedings of the 25th International Conference on Machine Learning, pp. 848-855, ACM (2008) · Zbl 1073.62547
[22] Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B 58, 267-288 (1996) · Zbl 0850.62538
[23] Tseng, P., Yun, S.: A coordinate gradient descent method for nonsmooth separable minimization. Math. Program. 117, 387-423 (2009) · Zbl 1166.90016 · doi:10.1007/s10107-007-0170-0
[24] Tseng, P.: Approximation accuracy, gradient methods, and error bound for structured convex optimization. Math. Program. 125(2), 263-295 (2010) · Zbl 1207.65084 · doi:10.1007/s10107-010-0394-2
[25] Van den Berg, E., Schmidt, M., Friedlander M., Murphy, K.: Group Sparsity Via Linear-Time Projection. Technical Report TR-2008-09, Department of Computer Science, University of British Columbia (2008)
[26] Wright, S., Nowak, R., Figueiredo, M.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57(7), 2479-2493 (2009) · Zbl 1391.94442 · doi:10.1109/TSP.2009.2016892
[27] Yang, H., Xu, Z., King, I., Lyu, M.: Online learning for group Lasso. In: 27th International Conference on, Machine Learning (2010) · Zbl 0634.90064
[28] Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 68(1), 49-67 (2006) · Zbl 1141.62030 · doi:10.1111/j.1467-9868.2005.00532.x
[29] Zheng, H., Chen, S., Mo, Z., Huang, X.: Numerical Computation (in Chinese). Wuhan University Press, Wuhan (2004)
[30] Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. B. 67(2), 301-320 (2005) · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x
[31] Zou, H.: The adaptive Lasso and its oracle properties. J. Am. Stat. Assoc. 101(476), 1418-1429 (2006) · Zbl 1171.62326 · doi:10.1198/016214506000000735
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.