×

zbMATH — the first resource for mathematics

Variable selection in a class of single-index models. (English) Zbl 1230.62062
Summary: We discuss variable selection in a class of single-index models in which we do not assume the error term as additive. Following the idea of sufficient dimension reduction, we first propose a unified method to recover the direction, then reformulate it under the least square framework. Differing from many other existing results associated with nonparametric smoothing methods for density function, the bandwidth selection in our proposed kernel function essentially has no impact on its root-\(n\) consistency or asymptotic normality. To select the important predictors, we suggest using the adaptive lasso method which is computationally efficient. Under some regularity conditions, the adaptive lasso method enjoys the oracle property in a general class of single-index models. In addition, the resulting estimation is shown to be asymptotically normal, which enables us to construct a confidence region for the estimated direction. The asymptotic results are augmented through comprehensive simulations, and illustrated by an analysis of air pollution data.

MSC:
62G08 Nonparametric regression and quantile regression
62H12 Estimation in multivariate analysis
65C60 Computational problems in statistics (MSC2010)
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Altham P.M.E. (1984) Improving the precision of estimation by fitting a generalized linear model and quasi-likelihood. Journal of the Royal Statistical Society: Series B 46: 118–119
[2] Carroll R.J., Fan J., Gijbels I., Wand M.P. (1997) Generalized partially linear single-index models. Journal of the American Statistical Association 92: 477–489 · Zbl 0890.62053 · doi:10.1080/01621459.1997.10474001
[3] Chen C.H., Li K.C. (1998) Can SIR be as popular as multiple linear regression?. Statistica Sinica 8: 289–316 · Zbl 0897.62069
[4] Cook R.D. (1998) Regression graphics: Ideas for studying regressions through graphics. Wiley & Sons, New York · Zbl 0903.62001
[5] Cook R.D. (2004) Testing predictor contributions in sufficient dimension reduction. Annals of Statistics 32: 1061–1092 · Zbl 1092.62046
[6] Cook R.D., Ni L. (2005) Sufficient dimension reduction via inverse regression: A minimum discrepancy approach. Journal of the American Statistical Association 100: 410–428 · Zbl 1117.62312 · doi:10.1198/016214504000001501
[7] Cook R.D., Weisberg S. (1991) Discussion to ”Sliced inverse regression for dimension reduction”. Journal of the American Statistical Association 86: 316–342 · Zbl 1353.62037 · doi:10.1080/01621459.1991.10475035
[8] Donoho D.L., Johnstone I.M. (1994) Ideal spatial adaptation by wavelet shrinkage. Biometrika 81: 425–455 · Zbl 0815.62019 · doi:10.1093/biomet/81.3.425
[9] Efron B., Hastie T., Johnstone I., Tibshirani R. (2004) Least angle regression. Annals of Statistics 32: 407–499 · Zbl 1091.62054 · doi:10.1214/009053604000000067
[10] Fan J.Q., Li R. (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96: 1348–1360 · Zbl 1073.62547 · doi:10.1198/016214501753382273
[11] Fan J.Q., Peng H. (2004) Nonconcave penalized likelihood with a diverging number of parameters. Annals of Statistics 32: 928–961 · Zbl 1092.62031 · doi:10.1214/009053604000000256
[12] Geyer C. (1994) On the asymptotics of constraint M-estimation. Annals of Statistics 22: 1993–2010 · Zbl 0829.62029 · doi:10.1214/aos/1176325768
[13] Hall P., Li K.C. (1993) On almost linearity of low dimensional projection from high dimensional data. Annals of Statistics 21: 867–889 · Zbl 0782.62065 · doi:10.1214/aos/1176349155
[14] Härdle W., Hall P., Ichimura H. (1993) Optimal smoothing in single-index models. Annals of Statistics 21: 157–178 · Zbl 0770.62049 · doi:10.1214/aos/1176349020
[15] Härdle W., Mammen E. (1993) Comparing nonparametric versus parametric regression fits. Annals of Statistics 21: 1926–1947 · Zbl 0795.62036 · doi:10.1214/aos/1176349403
[16] Härdle W., Stoker T.M. (1989) Investigating smooth multiple regression by the method of average derivatives. Journal of the American Statistical Association 84: 986–995 · Zbl 0703.62052
[17] Hristache M., Juditsky A., Spokoiny V. (2001) Direct estimation of the index coefficient in a single-index model. Annals of Statistics 29: 595–623 · Zbl 1012.62043 · doi:10.1214/aos/1009210681
[18] Huber P.J. (1981) Robust statistics. Wiley & Sons, New York
[19] Knight K., Fu W.J. (2000) Asymptotics for lasso-type estimators. Annals of Statistics 28: 1356–1378 · Zbl 1105.62357 · doi:10.1214/aos/1015957397
[20] Kong E., Xia Y.C. (2007) Variable selction for the single-index model. Biometrika 94: 217–229 · Zbl 1142.62353 · doi:10.1093/biomet/asm008
[21] Li K.C. (1991) Sliced inverse regression for dimension reduction (with discussion). Journal of the American Statistical Association 86: 316–342 · Zbl 0742.62044 · doi:10.1080/01621459.1991.10475035
[22] Li K.C., Duan N.H. (1989) Regression analysis under link violation. Annals of Statistics 17: 1009–1052 · Zbl 0753.62041 · doi:10.1214/aos/1176347254
[23] Li L.X. (2007) Sparse sufficient dimension reduction. Biometrika 92: 603–613 · Zbl 1135.62062 · doi:10.1093/biomet/asm044
[24] Li L.X., Nachtsheim C.J. (2006) Sparse sliced inverse regression. Technometrics 48: 503–510 · doi:10.1198/004017006000000129
[25] NiL. Cook R.D., Tsai C.L. (2005) A note on shrinkage sliced inverse regression. Biometrika 92: 242–247 · Zbl 1068.62080 · doi:10.1093/biomet/92.1.242
[26] Powell J.L., Stock J.H., Stoker T.M. (1989) Semiparametric estimation of index coeffcients. Econometrika 57: 1403–1430 · Zbl 0683.62070 · doi:10.2307/1913713
[27] Serfling R.J. (1980) Approximation theorems of mathematical statistics. John Wiley & Sons Inc., New York · Zbl 0538.62002
[28] Stein C. (1981) Estimation the mean of a multivariate normal distribution. Annals of Statistics 9: 1135–1151 · Zbl 0476.62035 · doi:10.1214/aos/1176345632
[29] Wang H., Leng C.L. (2007) Unified lasso estimation via least square approximation. Journal of the American Statistical Association 102: 1039–1048 · Zbl 1306.62167 · doi:10.1198/016214507000000509
[30] Wang H., Li R., Tsai C.L. (2007) Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika 94: 553–568 · Zbl 1135.62058 · doi:10.1093/biomet/asm053
[31] Xia Y.C., Tong H., Li W.K., Zhu L.X. (2002) An adaptive estimation of optimal regression subspace. Journal of the Royal Statistical Society: Series B 64: 363–410 · Zbl 1091.62028
[32] Zhu L.X., Fang K.T. (1996) Asymptotics for the kernel estimates of sliced inverse regression. Annals of Statistics 24: 1053–1067 · Zbl 0864.62027 · doi:10.1214/aos/1032526955
[33] Zhu L.P., Zhu L.X. (2007) On kernel method for sliced average variance estimation. Journal of Multivariate Analysis 98: 970–991 · Zbl 1113.62044 · doi:10.1016/j.jmva.2006.11.005
[34] Zhu L.P., Zhu L.X. (2009) Nonconcave penalized inverse regression in single-index models with high dimensional predictors. Journal of Multivariate Analysis 100: 862–875 · Zbl 1157.62037 · doi:10.1016/j.jmva.2008.09.003
[35] Zou H. (2006) The adaptive Lasso and its oracle properties. Journal of the American Statistical Association 101: 1418–1429 · Zbl 1171.62326 · doi:10.1198/016214506000000735
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.