×

Additive monotone regression in high and lower dimensions. (English) Zbl 1469.62234

Authors’ abstract: In numerous problems where the aim is to estimate the effect of a predictor variable on a response, one can assume a monotone relationship. For example, dose-effect models in medicine are of this type. In a multiple regression setting, additive monotone regression models assume that each predictor has a monotone effect on the response. In this paper, we present an overview and comparison of very recent frequentist methods for fitting additive monotone regression models. Three of the methods we present can be used both in the high dimensional setting, where the number of parameters \(p\) exceeds the number of observations \(n\), and in the classical multiple setting where \(1 < p \leq n\). However, many of the most recent methods only apply to the classical setting. The methods are compared through simulation experiments in terms of efficiency, prediction error and variable selection properties in both settings, and they are applied to the Boston housing data. We conclude with some recommendations on when the various methods perform best.

MSC:

62G08 Nonparametric regression and quantile regression
62P20 Applications of statistics to economics

Software:

bnpmr; gamair; scar; mboost
PDFBibTeX XMLCite
Full Text: DOI Euclid

References:

[1] Antoniadis, A., Bigot, J., and Gijbels, I. (2007). Penalized wavelet monotone regression. Statistics & Probability Letters, 77(16):1608-1621. · Zbl 1127.62034 · doi:10.1016/j.spl.2007.03.041
[2] Bacchetti, P. (1989). Additive isotonic models. Journal of the American Statistical Association, 84(405):289-294.
[3] Barlow, R. E., Bartholomew, D. J., Bremner, J. M., and Brunk, H. D. (1972). Statistical Inference Under Order Restrictions: The Theory and Application of Isotonic Regression. Hoboken, NJ: Wiley. · Zbl 0246.62038
[4] Barlow, R. E. and Brunk, H. D. (1972). The isotonic regression problem and its dual. Journal of the American Statistical Association, 67(337):140-147. · Zbl 0236.62050 · doi:10.1080/01621459.1972.10481216
[5] Bergersen, L. C., Tharmaratnam, K., and Glad, I. K. (2014). Monotone splines lasso. Computational Statistics & Data Analysis, 77:336-351. · Zbl 1506.62021 · doi:10.1016/j.csda.2014.03.013
[6] Bollaerts, K., Eilers, P. H., and Mechelen, I. (2006). Simple and multiple P-splines regression with shape constraints. British Journal of Mathematical and Statistical Psychology, 59(2):451-469.
[7] Bornkamp, B. and Ickstadt, K. (2008). Bayesian nonparametric estimation of continuous monotone functions with applications to dose-response analysis. Biometrics, 65(1):198-205. · Zbl 1159.62023 · doi:10.1111/j.1541-0420.2008.01060.x
[8] Bornkamp, B., Ickstadt, K., and Dunson, D. (2010). Stochastically ordered multiple regression. Biostatistics, 11(3):419-431. · Zbl 1437.62400
[9] Brezger, A. and Steiner, W.J. (2008). Monotonic regression based on Bayesian P-splines: an application to estimating price response functions from store-level scanner data. Journal of business & economic statistics, 26(1):90 -104.
[10] Bühlmann, P. and Yu, B. (2006). Sparse boosting. The Journal of Machine Learning Research, 7:1001-1024. · Zbl 1222.68155
[11] Chen, Y. and Samworth, R. J. (2016). Generalized additive and index models with shape constraints. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78(4):729 - 754. · Zbl 1414.62153 · doi:10.1111/rssb.12137
[12] Chipman, H. A., George, E., I, McCulloch, R. E. and Shively, T. S. (2016). High-dimensional nonparametric monotone function estimation using BART. arXiv preprint arXiv:1612.01619.
[13] Chiquet, J., Grandvalet, Y., and Charbonnier, C. (2012). Sparsity with sign-coherent groups of variables via the cooperative-lasso. The Annals of Applied Statistics, 6(2):795-830. · Zbl 1243.62101 · doi:10.1214/11-AOAS520
[14] Dette, H. and Scheder, R. (2006). Strictly monotone and smooth nonparametric regression for two or more variables. Canadian Journal of Statistics, 34(4):535-561. · Zbl 1115.62039 · doi:10.1002/cjs.5550340401
[15] Du, P., Cheng, G. and Liang, H. (2012). Semiparametric regression models with additive nonparametric components and high dimensional parametric components. Computational Statistics & Data Analysis, 56(6):2006-2017. · Zbl 1243.62053 · doi:10.1016/j.csda.2011.12.007
[16] Du, P., Parmeter, C. F. and Racine, J. S. (2013). Nonparametric kernel regression with multiple predictors and multiple shape constraints Statistica Sinica, 23(3):1347-1371. · Zbl 06202710
[17] Eilers, P. H. and Marx, B. D. (1996). Flexible smoothing with B-splines and penalties. Statistical Science, 11:89-102. · Zbl 0955.62562 · doi:10.1214/ss/1038425655
[18] Fang, Z. and Meinshausen, N. (2012). Lasso isotone for high-dimensional additive isotonic regression. Journal of Computational and Graphical Statistics, 21(1):72-91.
[19] Gunter, L., Zhu, J. and Murphy, S. (2007). Variable selection for optimal decision making. Conference on Artificial Intelligence in Medicine in Europe, 2007:149-154. Springer.
[20] Guo, J., Tang, M., Tian, M. and Zhu, K. (2013). Variable selection in high-dimensional partially linear additive models for composite quantile regression Computational Statistics & Data Analysis, 65:56-67. · Zbl 1471.62081 · doi:10.1016/j.csda.2013.03.017
[21] Harrison, D. and Rubinfeld, D. L. (1978). Hedonic housing prices and the demand for clean air. Journal of Environmental Economics and Management, 5(1):81-102. · Zbl 0375.90023 · doi:10.1016/0095-0696(78)90006-2
[22] Hastie, T. and Tibshirani, R. (1986). Generalized additive models. Statistical Science, 1(3):297-310. · Zbl 0645.62068 · doi:10.1214/ss/1177013604
[23] He, X. and Shi, P. (1998). Monotone B-spline smoothing. Journal of the American statistical Association, 93(442):643-650. · Zbl 1127.62322
[24] Hesterberg, T., Choi, N. H., Meier, L., and Fraley, C. (2008). Least angle and \(L_1\) penalized regression: A review. Statistics Surveys, 2:61-93. · Zbl 1189.62070 · doi:10.1214/08-SS035
[25] Hofner, B., Kneib, T., and Hothorn, T. (2016). A unified framework of constrained regression. Statistics and Computing, 26(1-2):1-14. · Zbl 1342.62115 · doi:10.1007/s11222-014-9520-y
[26] Hothorn, T., Buehlmann, P., Kneib, T., Schmid, M., and Hofner, B. (2017). mboost: Model-Based Boosting. R package version 2.8-1.
[27] Härdle, W. and Liang, H. (2007). Partially linear models. In Statistical methods for biostatistics and related fields, 87-103. Springer, Berlin, Heidelberg.
[28] Leitenstorfer, F. and Tutz, G. (2006). Generalized monotonic regression based on B-splines with an application to air pollution data. Biostatistics, 8(3):654-673. · Zbl 1118.62125 · doi:10.1093/biostatistics/kxl036
[29] Lian, H., Liang, H. and Ruppert, D. (2015). Separation of covariates into nonparametric and parametric parts in high-dimensional partially linear additive models. Statistica Sinica, 25(2):591-607. · Zbl 06503812
[30] Lin, L. and Dunson, D. B. (2014). Bayesian monotone regression using Gaussian process projection Biometrika, 101(2):303-317. · Zbl 1452.62285 · doi:10.1093/biomet/ast063
[31] Lin, Y. and Zhang, H. H. (2006). Component selection and smoothing in multivariate nonparametric regression. The Annals of Statistics, 34(5):2272-2297. · Zbl 1106.62041 · doi:10.1214/009053606000000722
[32] Liu, X., Wang, L. and Liang, H. (2011). Estimation and variable selection for semiparametric additive partial linear models Statistica Sinica, 21(3):1225-1248. · Zbl 1223.62020 · doi:10.5705/ss.2009.140
[33] Lou, Y., Bien, J., Caruana, R. and Gehrke, J. (2016). Sparse partially linear additive models. Journal of Computational and Graphical Statistics,25(4): 1126-1140.
[34] Luss, R., Rosset, S., and Shahar, M. (2012). Efficient regularized isotonic regression with application to gene-gene interaction search. The Annals of Applied Statistics, 6(1):253-283. · Zbl 1235.62046 · doi:10.1214/11-AOAS504
[35] Lv, J., Yang, H. and Guo, C. (2017). Variable selection in partially linear additive models for modal regression Communications in Statistics-Simulation and Computation 46(7): 5646 - 5665. · Zbl 1380.62196 · doi:10.1080/03610918.2016.1171346
[36] Meyer, M. C. (2008). Inference using shape-restricted regression splines. The Annals of Applied Statistics, 2(3):1013-1033. · Zbl 1149.62033 · doi:10.1214/08-AOAS167
[37] Meyer, M. C. (2013). Semi-parametric additive constrained regression. Journal of Nonparametric Statistics, 25(3):715-730. · Zbl 1416.62223 · doi:10.1080/10485252.2013.797577
[38] Meyer, M. C., Hackstadt, A. J., and Hoeting, J. A. (2011). Bayesian estimation and inference for generalised partial linear models using shape-restricted splines. Journal of Nonparametric Statistics, 23(4):867-884. · Zbl 1230.62054 · doi:10.1080/10485252.2011.597852
[39] Pya, N. and Wood, S. N. (2015). Shape constrained additive models. Statistics and Computing, 25(3):543-559. · Zbl 1331.62367 · doi:10.1007/s11222-013-9448-7
[40] Ramsay, J. O. (1988). Monotone regression splines in action. Statistical Science, 3(4):425-441.
[41] Ruppert, D. (2002). Selecting the number of knots for penalized splines. Journal of Computational and Graphical Statistics, 11(4):735-757.
[42] Saarela, O. and Arjas, E. (2011). A method for Bayesian monotonic multiple regression. Scandinavian Journal of Statistics, 38(3):499-513. · Zbl 1246.62111
[43] Schell, M. J. and Singh, B. (1997). The reduced monotonic regression method. Journal of the American Statistical Association, 92(437):128-135. · Zbl 0890.62035 · doi:10.1080/01621459.1997.10473609
[44] Tutz, G. and Leitenstorfer, F. (2007). Generalized smooth monotonic regression in additive modeling. Journal of Computational and Graphical Statistics, 16(1):165-188. · Zbl 1118.62125 · doi:10.1093/biostatistics/kxl036
[45] Wang, L. and Xue, L. (2015). Constrained polynomial spline estimation of monotone additive models. Journal of Statistical Planning and Inference, 167:27-40. · Zbl 1327.62230 · doi:10.1016/j.jspi.2015.06.001
[46] Wei, F. (2012). Group selection in high-dimensional partially linear additive models Brazilian Journal of Probability and Statistics, 26(3): 219-243. · Zbl 1239.62048 · doi:10.1214/10-BJPS129
[47] Wood, S. N. (2006). Generalized additive models: an introduction with R. Chapman and Hall/CRC. · Zbl 1087.62082
[48] Zhang, H. H., Cheng, G. and Liu, Y. (2011). Linear or nonlinear? Automatic structure discovery for partially linear models Journal of the American Statistical Association, 106(495):1099-1112. · Zbl 1229.62051 · doi:10.1198/jasa.2011.tm10281
[49] Zou H, Hastie T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2):301-20. · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.