zbMATH — the first resource for mathematics

The effect of estimating weights in weighted least squares. (English) Zbl 0691.62061
When doing weighted least squares regression the weights are most often estimated, but standard packages pretend they are known in advance. The effects of estimating weights for small-to-moderate sample sizes may not be negligible. This paper explores this question and indicates how the bootstrap might be used. The model $y_ i=\chi^ t_ i\beta +\delta_ i,\quad \delta_ i=\sigma g(\chi^ t_ i\beta,\quad \theta,\quad \chi_ i)\epsilon_ i,$ is considered, where the $$\delta_ i$$ errors are assumed to be independent with mean 0, and $$\epsilon_ i$$ have variance 1. The effects of the initial estimate of $$\beta$$, a method for estimating $$\theta$$, and the number of iterative weightings on generalized least squares are studied. In general, $$c\geq 3$$ iterations are necessary for covariance stabilization, although $$c\geq 2$$ suffice if either (a) the variance does not depend on the mean and the errors are symmetrically distributed, or (b) unweighted least squares is used as the initial estimate. If both (a) and (b) hold, $$c=1$$ iteration suffices for covariance stabilization of the variance function. There is no optimal number of iterations that suffices for all problems. Using a starting estimate other than unweighted least squares can result in a change in the covariance expansion. The bootstrap method used there is appropriate when the errors are known to be i.i.d. Otherwise, techniques such as those proposed by the second author [Ann. Stat. 14, 1261-1295 (1986; Zbl 0618.62072)] might be used.
Reviewer: Guijing Chen

MSC:
 62J05 Linear regression; mixed models 62F10 Point estimation
Full Text: