zbMATH — the first resource for mathematics

Second-order approximation of the entropy in nonlinear least-squares estimation. (English) Zbl 0812.62071
Kybernetika 30, No. 2, 187-198 (1994); erratum ibid. 32, No. 1, 104 (1996).
Summary: Measures of variability of the least-squares estimator \(\widehat {\theta}\) are essential to assess the quality of the estimation. In nonlinear regression, an accurate approximation of the covariance matrix of \(\widehat {\theta}\) is difficult to obtain. A second-order approximation of the entropy of the distribution of \(\widehat {\theta}\) is proposed, which is only slightly more complicated than the widely used bias approximation of M. J. Box [J. R. Stat. Soc, Ser. B 33, 171- 201 (1971; Zbl 0232.62029)]. It is based on the “flat” or “saddle- point approximation” of the density of \(\widehat {\theta}\). The neglected terms are of order \({\mathcal O} (\sigma^ 4)\), while the classical first order approximation neglects terms of order \({\mathcal O} (\sigma^ 2)\). Various illustrative examples are presented, including the use of the approximate entropy as a criterion for experimental design.

62J02 General nonlinear regression
62E17 Approximations to statistical distributions (nonasymptotic)
62E20 Asymptotic distribution theory in statistics
62K05 Optimal statistical designs
Full Text: Link EuDML
[1] S. Amari: Differential-Geometrical Methods in Statistics. Springer, Berlin 1985. · Zbl 0559.62001
[2] D. Bates, D. Watts: Relative curvature measures of nonlinearity. J. Roy. Statist. Soc. Ser. B 42 (1980), 1-25. · Zbl 0455.62028
[3] M. Box: Bias in nonlinear estimation. J. Roy. Statist. Soc. Ser. B 33 (1971), 171-201. · Zbl 0232.62029
[4] G. Clarke: Moments of the least-squares estimators in a non-linear regression model. J. Roy. Statist. Soc. Ser. B 42 (1980), 227-237. · Zbl 0436.62054
[5] P. Hougaard: Saddlepoint approximations for curved exponential families. Statist. Probab. Lett. 3 (1985), 161-166. · Zbl 0573.62016 · doi:10.1016/0167-7152(85)90055-0
[6] A. Pázman: Probability distribution of the multivariate nonlinear least-squares estimates. Kybernetika 20 (1984), 209-230. · Zbl 0548.62043 · eudml:28735
[7] A. Pázman: Small-sample distributional properties of nonlinear regression estimators (a geometric approach) (with discussion). Statistics 21 (1990), 3, 323-367. · Zbl 0724.62063
[8] N. Reid: Saddlepoint methods and statistical inference. Statist. Sci. 3 (1988), 213-238. · Zbl 0955.62541 · doi:10.1214/ss/1177012906
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.