\(l_1\) regularized multiplicative iterative path algorithm for non-negative generalized linear models. (English) Zbl 1466.62156

Mandal, B. N.; Ma, Jun

Comput. Stat. Data Anal. 101, 289-299 (2016).

Summary: In regression modeling, often a restriction that regression coefficients are non-negative is faced. The problem of model selection in non-negative generalized linear models (NNGLM) is considered using lasso, where regression coefficients in the linear predictor are subject to non-negative constraints. Thus, non-negatively constrained regression coefficient estimation is sought by maximizing the penalized likelihood (such as the \(l_1\)-norm penalty). An efficient regularization path algorithm is proposed for generalized linear models with non-negative regression coefficients. The algorithm uses multiplicative updates which are fast and simultaneous. Asymptotic results are also developed for the constrained penalized likelihood estimates. Performance of the proposed algorithm is shown in terms of computational time, accuracy of solutions and accuracy of asymptotic standard deviations.

MSC:

62-08	Computational methods for problems pertaining to statistics
62J12	Generalized linear models (logistic models)

Keywords:

generalized linear models; Lasso; elastic net; \(l_1\)-norm penalty; regularization path; non-negativity constraints

Software:

rpart; glmnet; ElemStatLearn

References:

[1]	Antoniadis, A.; Gijbels, I.; Nikolova, M., Penalized likelihood regression for generalized linear models with nonquadratic penalties, Ann. Inst. Statist. Math., 63, 585-615, (2011) · Zbl 1333.62113
[2]	Armijo, L., Minimization of functions having Lipschitz-continuous first partial derivatives, Pacific J. Math., 16, 1-3, (1966) · Zbl 0202.46105
[3]	Bertsekas, D. P., Projected Newton methods for optimization problems with simple constraints, SIAM J. Control Optim., 20, 221-246, (1982) · Zbl 0507.49018
[4]	Chan, R. H.; Ma, J., A multiplicative iterative algorithm for box-constrained penalized likelihood image restoration, IEEE Trans. Image Process., 21, 3168-3181, (2012) · Zbl 1373.94063
[5]	Franc, V., Hlaváč, V., Navara, M., 2005. Sequential coordinate-wise algorithm for the non-negative least squares problem. In: CAIP 3691, pp. 407-414.
[6]	Friedman, J.; Hastie, T.; Höfling, H.; Tibshirani, R., Pathwise coordinate optimization, Ann. Appl. Stat., 2, 1, 302-332, (2007) · Zbl 1378.90064
[7]	Friedman, J.; Hastie, T.; Tibshirani, R., Regularization paths for generalized linear models via coordinate descent, J. Stat. Softw., 33, 1, 1-22, (2010)
[8]	Hastie, T.; Tibshirani, R.; Friedman, J., The elements of statistical learning: prediction, inference and data mining, (2009), Springer-Verlag New York · Zbl 1273.62005
[9]	Honore, B. E.; Powell, J. L., Pairwise difference estimators of censored and truncated regression models, J. Econometrics, 64, 241-278, (1994) · Zbl 0808.62038
[10]	Lawson, C. L.; Hanson, R. J., Solving least squares problems, SIAM Rev., 18, 3, 518-520, (1995) · Zbl 0860.65029
[11]	Lipovetsky, S., Linear regression with special coefficient features attained via parameterization in exponential, logistic, and multinomial-logit forms, Math. Comput. Modelling, 49, 7, 1427-1435, (2009) · Zbl 1165.62327
[12]	Ma, J., Multiplicative algorithms for maximum penalized likelihood inversion with nonnegative constraints and generalized error distributions, Commun. Statist. Theor. Meth., 35, 5, 831-848, (2006) · Zbl 1093.62031
[13]	Ma, J., Positively constrained multiplicative iterative algorithm for maximum penalized likelihood tomographic reconstruction, IEEE Trans. Nucl. Sci., 57, 181-192, (2010)
[14]	Ma, J., Couturier, D., Heritier, S., Marschner, I., 2014a. Penalized likelihood estimation for semiparametric proportional hazard models with interval censored data. Department of Statistics Report, Macquarie University, NSW, Australia.
[15]	Ma, J.; Heritier, S.; Lô, S., On the maximum penalized likelihood approach for proportional hazard models with right censored survival data, Comput. Statist. Data Anal., 74, 142-156, (2014) · Zbl 1506.62121
[16]	McCullagh, P.; Nelder, J., Generalized linear models, (1989), CHAPMAN & HALL/CRC Boca Raton · Zbl 0744.62098
[17]	McDonald, J. W.; Diamond, I. D., On the Fitting of generalized linear models with nonnegativity parameter constraints, Biometrics, 201-206, (1990) · Zbl 0715.62135
[18]	Moore, T. J.; Sadler, B. M.; Kozick, R. J., Maximum-likelihood estimation, the cramér-Rao bound, and the method of scoring with parameter constraints, IEEE Trans. Signal Process., 56, 895-908, (2008) · Zbl 1390.94310
[19]	Ostrowski, J. M., Solution of equations and system of equations, (1966), Academic New York · Zbl 0222.65070
[20]	Park, M.; Hastie, T., \(l_1\)-regularization path algorithm for generalized linear models, J. R. Stat. Soc. Ser. B Stat. Methodol., 69, 659-677, (2007) · Zbl 07555370
[21]	Sha, F.; Lin, Y., Multiplicative updates for nonnegative quadratic programming, Neural Comput., 19, 2004-2031, (2007) · Zbl 1161.90456
[22]	Therneau, T., Atkinson, B., Ripley, B., 2014. rpart: Recursive Partitioning and Regression Trees. R package version 4.1-8. URL: http://CRAN.R-project.org/package=rpart.
[23]	Tibshirani, R., Regression shrinkage and selection via the lasso, J. R. Stat. Soc. Ser. B Stat. Methodol., 58, 267-288, (1996) · Zbl 0850.62538
[24]	Tibshirani, R.; Saunders, M.; Rosset, S.; Zhu, J.; Knight, K., Sparsity and smoothness via the fused lasso, J. R. Stat. Soc. Ser. B Stat. Methodol., 67, 91-108, (2005) · Zbl 1060.62049
[25]	Wu, L.; Yang, Y.; Liu, H., Nonnegative-lasso and application in index tracking, Comput. Statist. Data Anal., 70, 116-126, (2013) · Zbl 1471.62220
[26]	Zou, H.; Hastie, T., Regularization and variable selection via the elastic net, J. R. Stat. Soc. Ser. B, 67, 301-320, (2005) · Zbl 1069.62054