On the degrees of freedom in shape-restricted regression. (English) Zbl 1105.62340

Summary: For the problem of estimating a regression function, \(\mu\) say, subject to shape constraints, like monotonicity or convexity, it is argued that the divergence of the maximum likelihood estimator provides a useful measure of the effective dimension of the model. Inequalities are derived for the expected mean squared error of the maximum likelihood estimator and the expected residual sum of squares. These generalize equalities from the case of linear regression. As an application, it is shown that the maximum likelihood estimator of the error variance \(\sigma^2\) is asymptotically normal with mean \(\sigma^2\) and variance \(2\sigma_2/n\). For monotone regression, it is shown that the maximum likelihood estimator of \(\mu\) attains the optimal rate of convergence, and a bias correction to the maximum likelihood estimator of \(\sigma^2\) is derived.


62G08 Nonparametric regression and quantile regression
62F12 Asymptotic properties of parametric estimators
Full Text: DOI


[1] Breiman, L. (1968). Probability. Addison-Weseley, Reading, MA. · Zbl 0174.48801
[2] Donoho, D. (1990). Gelfand n-widths and the method of least squares. Unpublished manuscript.
[3] Donoho, D. and Johnstone, I. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200-1225. JSTOR: · Zbl 0869.62024
[4] Efromovich, S. (1997). On quasi-linear wavelet estimation. J. Amer. Statist. Assoc. 94 189-204. JSTOR: · Zbl 1072.62557
[5] Feller, W. (1971). An Introduction to Probability Theory and Its Applications 2. Wiley, New York. · Zbl 0219.60003
[6] Gasser, T. Sroka, L. and Jennen-Steinmetz, C. (1986). Residual variance and residual pattern in non-linear regression. Biometrika 73 625-633. JSTOR: · Zbl 0649.62035
[7] Groeneboom, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer (L. Le Cam and R. Olshen, eds.) 2 539-558. Wadsworth and Brooks/Cole, Belmont, CA. · Zbl 1373.62144
[8] Groeneboom, P. (1989). Brownian motion with a parabolic drift and airy functions. Probab. Theory Related Fields 81 79-109.
[9] Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London. · Zbl 0747.62061
[10] Klass, M. (1983). On the maximum of a random walkwith a small negative drift. Ann. Probab. 11 491-505. · Zbl 0514.60069
[11] Meyer, M. (1996). Shape restricted inference with applications to nonparametric regression, smooth nonparametric regression, and density estimation. Ph.D. thesis, Dept. statistics, Univ. Michigan.
[12] Meyer, M. and Woodroofe, M. (1998). Variance estimation in monotone regression. Technical Report 322, Dept. Statistics, Univ. Michigan.
[13] Rice, J. (1984). Bandwidth choice for nonparametric regression. Ann. Statist. 12 1215-1230. · Zbl 0554.62035
[14] Robertson, T., Wright, F. and Dykstra, R. (1988). Order Restricted Inference, Wiley, New York. · Zbl 0645.62028
[15] Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135-1151. · Zbl 0476.62035
[16] Van der Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907-924. · Zbl 0709.62040
[17] Woodroofe, M. (1982). Non-linear Renewal Theory in Sequential Analysis. SIAM, Philadelphia. · Zbl 0487.62062
[18] Wright, F. T. (1981). The asymptotic behavior of monotone regression estimates. Ann. Statist. 9 449-453. · Zbl 0471.62062
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.