×

Multiclass-penalized logistic regression. (English) Zbl 07476345

Summary: A multinomial logistic regression model that penalizes the number of class-specific parameters is proposed. The number of parameters in a standard multinomial regression model increases linearly with the number of classes and number of explanatory variables. The multiclass-penalized regression model clusters parameters together by penalizing the differences between class-specific parameter vectors, instead of penalizing the number of explanatory variables. The model provides interpretable parameter estimates, even in settings with many classes. An algorithm for maximum likelihood estimation in the multiclass-penalized regression model is discussed. Applications to simulated and real data show in- and out-of-sample improvements in performance relative to a standard multinomial regression model.

MSC:

62-XX Statistics

Software:

glmnet; msgl
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Boyd, S.; Parikh, N.; Chu, E.; Peleato, B.; Eckstein, J., Distributed optimization and statistical learning via the alternating direction method of multipliers, (Foundations and Trends® in Machine Learning, Vol. 3 (2011)), 1-122 · Zbl 1229.90122
[2] Carson, R. T.; Louviere, J. J., Statistical properties of consideration sets, J. Choice Model., 13, 37-48 (2014)
[3] Chi, E. C.; Lange, K., Splitting methods for convex clustering, J. Comput. Graph. Stat., 24, 994-1013 (2015)
[4] Cramer, J. S.; Ridder, G., Pooling states in the multinomial logit model, J. Econom., 47, 267-272 (1991) · Zbl 1359.62314
[5] Faraway, J. J., Faraway: functions and datasets for books by julian faraway (2016)
[6] Friedman, J.; Hastie, T.; Tibshirani, R., Regularization paths for generalized linear models via coordinate descent, J. Stat. Softw., 33, 1 (2010)
[7] Nibbering, D., A high-dimensional multinomial choice model (2019), Working paper
[8] Price, B. S.; Geyer, C. J.; Rothman, A. J., Automatic response category combination in multinomial logistic regression, J. Comput. Graph. Stat., 1-9 (2019) · Zbl 07499092
[9] Simon, N.; Friedman, J.; Hastie, T., A blockwise descent algorithm for group-penalized multiresponse and multinomial regression (2013), Preprint
[10] Taddy, M., Distributed multinomial regression, Ann. Appl. Stat., 9, 1394-1414 (2015) · Zbl 1454.62036
[11] Tibshirani, R., Regression shrinkage and selection via the lasso, J. R. Stat. Soc., Ser. B, Methodol., 267-288 (1996) · Zbl 0850.62538
[12] Tutz, G.; Pößnecker, W.; Uhlmann, L., Variable selection in general multinomial logit models, Comput. Stat. Data Anal., 82, 207-222 (2015) · Zbl 1507.62170
[13] Vincent, M.; Hansen, N. R., Sparse group lasso and high dimensional multinomial classification, Comput. Stat. Data Anal., 71, 771-786 (2014) · Zbl 1471.62200
[14] Yuan, M.; Lin, Y., Model selection and estimation in regression with grouped variables, J. R. Stat. Soc., Ser. B, Stat. Methodol., 68, 49-67 (2006) · Zbl 1141.62030
[15] Zanutto, E. L.; Bradlow, E. T., Data pruning in consumer choice models, Quant. Mark. Econ., 4, 267-287 (2006)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.