zbMATH — the first resource for mathematics

A new class of biased estimates in linear regression. (English) Zbl 0784.62065
Summary: Consider the linear regression model \(Y=X \beta+ \varepsilon\), \(E \varepsilon=0\) and \(\text{Cov} (\varepsilon)=\sigma^ 2 I\). Motivated by an interpretation of ridge estimates \(\hat\beta_ R=(X' X+kI)^{-1}X' Y\), we propose a new class of biased estimates \(\hat\beta_ d=(X' X+I)^{-1}(X' Y+d \hat\beta)\) to combat multicollinearity, where \(0<d<1\) is a parameter and \(\hat\beta\) is the least squares estimate. \(\hat\beta_ d\) combines the advantages of \(\hat\beta_ R\) and Stein estimate \(\hat\beta_ s=c\hat\beta\).
Theory and simulation results show that \(\hat\beta_ d\) has similarly good properties as \(\hat\beta_ R\). The advantage of \(\hat\beta_ d\) over \(\hat\beta_ R\) is that \(\hat\beta_ d\) is a linear function of \(d\). So the selection of \(d\) is simple.

62J07 Ridge regression; shrinkage estimators (Lasso)
62J05 Linear regression; mixed models
Full Text: DOI
[1] Chatterjee S., Regression Analysis by Example (1977) · Zbl 0900.62356
[2] DOI: 10.2307/1268518 · Zbl 0461.62059
[3] DOI: 10.2307/1267351 · Zbl 0202.17205
[4] DOI: 10.2307/1267352 · Zbl 0202.17206
[5] DOI: 10.1080/03610927608827333 · Zbl 0326.62047
[6] James, W. and Stein, C. Estimation with quadratic loss. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability. Berkeley. pp.361–379. University of California Press. · Zbl 1281.62026
[7] Malinvard E., Statistical Methods of Econometrics (1980)
[8] DOI: 10.2307/1267380 · Zbl 0269.62061
[9] DOI: 10.2307/2285832 · Zbl 0319.62049
[10] Stein, C. Inadmissibility of usual estimator for the mean of a multivariate Normal distribution. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability. Berkeley. pp.197–206. University of California Press.
[11] Wang Songgui, The theory and application of linear model(in Chinese) (1987) · Zbl 0671.62063
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.