×

Positive shrinkage, improved pretest and absolute penalty estimators in partially linear models. (English) Zbl 1160.62066

Summary: Shrinkage estimators of a partially linear regression parameter vector are constructed by shrinking estimators in the direction of the estimate which is appropriate when the regression parameters are restricted to a linear subspace. We investigate the asymptotic properties of positive Stein-type and improved pretest semiparametric estimators under quadratic loss. Under an asymptotic distributional quadratic risk criterion, their relative dominance picture is explored analytically.
It is shown that positive Stein-type semiparametric estimators perform better than the usual Stein-type and least square semiparametric estimators and that an improved pretest semiparametric estimator is superior to the usual pretest semiparametric estimator. We also consider an absolute penalty type estimator for partially linear models and give Monte Carlo simulation comparisons of positive shrinkage, improved pretest and the absolute penalty type estimators. The comparisons show that the shrinkage method performs better than the absolute penalty type estimation method when the dimension of the parameter space is much larger than that of the linear subspace.

MSC:

62J07 Ridge regression; shrinkage estimators (Lasso)
62G08 Nonparametric regression and quantile regression
62H12 Estimation in multivariate analysis
62E20 Asymptotic distribution theory in statistics
62G05 Nonparametric estimation
65C05 Monte Carlo methods

Software:

R; PDCO; sm
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Ahmed, S. E.; Ullah, B., Improved biased estimation in an ANOVA model, Linear Algebra Appl., 289, 3-24 (1999) · Zbl 0930.62069
[2] Ahmed, S. E., Shrinkage estimation of regression coefficients from censored data with multiple observations, (Ahmed, S. E., Empirical Bayes and Likelihood Inference. Empirical Bayes and Likelihood Inference, Lecture Notes in Statistics, vol. 148 (2001), Springer-Verlag: Springer-Verlag New York), 103-120
[3] Ahmed, S. E.; Doksum, K. A.; Hossain, S.; You, J., Shrinkage, pretest and absolute penalty estimators in partially linear models, Aust. New Zealand J. Stat., 49, 4, 435-454 (2007) · Zbl 1158.62029
[4] Bancroft, T. A., On biases in estimation due to the use of preliminary test of significance, The Ann. Math. Stat., 15, 190-204 (1944) · Zbl 0063.00180
[5] Bunea, F., Consistent covariate selection and post model selection inference in semiparametric regression, Ann. Stat., 32, 898-927 (2004) · Zbl 1092.62045
[6] Chen, H., Convergence rates for parametric components in a partially linear model, Ann. Stat., 16, 136-147 (1988)
[7] Chen, H.; Shiau, J., Data-driven efficient estimation for a partially linear model, Ann. Stat., 22, 211-237 (1994) · Zbl 0806.62029
[8] S. Chen, D.L. Donoho, On Basis Pursuit, Technical Report, Department of Statistics, Stanford University, 1994.; S. Chen, D.L. Donoho, On Basis Pursuit, Technical Report, Department of Statistics, Stanford University, 1994.
[9] Chen, S.; Donoho, D. L.; Saunders, M. A., Atomic decomposition by basis pursuit, SIAM J. Sci. Comput., 20, 1, 33-61 (1999) · Zbl 0919.94002
[10] Donald, G.; Newey, K., Series estimation of semilinear models, J. Multivariate Anal., 50, 30-40 (1994) · Zbl 0798.62074
[11] Engle, R. F.; Granger, C. W.J.; Rice, J.; Weiss, A., Semiparametric estimates of the relation between weather and electricity sales, J. Am. Stat. Assoc., 80, 310-319 (1986)
[12] Eubank, R. L.; Hart, J. D.; Speckman, P., Trigonometric series regression estimators with an application to partially linear models, J. Multivariate Anal., 32, 70-83 (1990) · Zbl 0709.62041
[13] Fan, J.; Härdle, W.; Mammen, E., Direct estimation of low-dimensional components in additive models, Ann. Stat., 26, 943-971 (1998) · Zbl 1073.62527
[14] Gao, J. T., Asymptotic theory for partially linear models, Commun. Stat. - Theory Methods, 24, 1985-2009 (1995) · Zbl 0937.62592
[15] Gao, J. T., The laws of the iterated logarithm of some estimates in partially linear models, Stat. Prob. Lett., 25, 153-162 (1995) · Zbl 0837.62041
[16] Gao, J. T., Adaptive parametric test in a semiparametric regression model, Commun. Stat. - Theory Methods, 26, 787-800 (1997) · Zbl 0912.62043
[17] Hamilton, A.; Truong, K., Local linear estimation in partially linear models, J. Multivariate Anal., 60, 1-19 (1997) · Zbl 0883.62041
[18] Härdle, W.; Liang, H.; Gao, J., Partially Linear Models (2000), Physica-Verlag: Physica-Verlag Heidelberg
[19] Heckman, N., Spline smoothing in a partially linear model, J. Royal Stat. Soc.: Ser. B, 48, 244-248 (1986) · Zbl 0623.62030
[20] Ihaka, R.; Gentleman, R., R: a language for data analysis and graphics, J. Comput. Graph. Stat., 5, 299-314 (1996)
[21] Liang, H.; Härdle, W., Large sample theory of the estimation of the error distribution for a semiparametric model, Commun. Stat. - Theory Methods, 28, 2025-2037 (1999) · Zbl 0935.62048
[22] Liang, H.; Wong, S.; Robins, J. M.; Carroll, R. J., Estimation in partially linear models with missing covariates, J. Am. Stat. Assoc., 99, 357-367 (2004) · Zbl 1117.62385
[23] Rice, J., Convergence rates for partially splined models, Stat. Prob. Lett., 4, 203-208 (1986) · Zbl 0628.62077
[24] Robinson, P., Root-\(N\)-consistent semiparametric regression, Econometrica, 56, 931-954 (1988) · Zbl 0647.62100
[25] Saleh, A. K.Md. E., Theory of Preliminary Test and Stein-Type Estimation with Applications (2006), John Wiley & Sons · Zbl 1094.62024
[26] Schick, A., Estimation of the autocorrelation coefficient in the presence of a regression trend, Stat. Prob. Lett., 21, 371-380 (1994) · Zbl 0805.62044
[27] Schick, A., Efficient estimation in a semiparametric additive regression model with autoregressive errors, Stochastic Process. Appl., 61, 339-361 (1996) · Zbl 0844.62029
[28] Schick, A., An adaptive estimator of the autocorrelation coefficient in regression models with autoregressive errors, Stochastic Process. Appl., 19, 575-589 (1998) · Zbl 0928.62078
[29] Sclove, S. L.; Morris, C.; Radhakrishnan, R., Optimality of preliminary test estimation for the multinormal mean, Ann. Math. Stat., 43, 1481-1490 (1972) · Zbl 0249.62029
[30] Shi, J.; Lau, T. S., Empirical likelihood for partially linear models, J. Multivariate Anal., 72, 132-149 (2000) · Zbl 0978.62034
[31] Shi, P.; Li, G., A note of the convergence rates of \(m\)-estimates for partially linear models, Statistics, 26, 27-47 (1995) · Zbl 0812.62046
[32] Speckman, P., Kernel smoothing in partial linear models, J. Royal Stat. Soc.: Ser. B, 50, 413-437 (1988) · Zbl 0671.62045
[33] Tibshirani, R., Regression shrnikage and selection via the LASSO, J. Royal Stat. Soc.: Ser. B, 58, 267-288 (1996) · Zbl 0850.62538
[34] Wang, Q.; Linton, O.; Härdle, W., Semiparametric regression analysis with missing response at random, J. Am. Stat. Assoc., 99, 334-345 (2004) · Zbl 1117.62441
[35] Xue, H.; Lam, K. F.; Gouying, L., Sieve maximum likelihood estimator for semiparametric regression models with current status data, J. Am. Stat. Assoc., 99, 346-356 (2004) · Zbl 1117.62449
[36] Bowman, A. W.; Azzalini, A., Applied smoothing Techniques for Data Analysis: The Kernel Approach with S-Plus Illustrations (1997), Oxford University Press: Oxford University Press Oxford · Zbl 0889.62027
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.