The bias of isotonic regression. (English) Zbl 1436.62137

Summary: We study the bias of the isotonic regression estimator. While there is extensive work characterizing the mean squared error of the isotonic regression estimator, relatively little is known about the bias. In this paper, we provide a sharp characterization, proving that the bias scales as \(O(n^{-\beta /3})\) up to log factors, where \(1\leq \beta \leq 2\) is the exponent corresponding to Hölder smoothness of the underlying mean. Importantly, this result only requires a strictly monotone mean and that the noise distribution has subexponential tails, without relying on symmetric noise or other restrictive assumptions.


62G08 Nonparametric regression and quantile regression
62G20 Asymptotic properties of nonparametric inference
62G05 Nonparametric estimation
62E20 Asymptotic distribution theory in statistics


Full Text: DOI arXiv Euclid


[1] E. S. Andersen. On the fluctuations of sums of random variables ii., Mathematica Scandinavica, 2:194-222, Dec. 1954. URL https://www.mscand.dk/article/view/10407. · Zbl 0058.12103
[2] M. Banerjee, C. Durot, and B. Sen. Divide and conquer in nonstandard problems and the super-efficiency phenomenon., Ann. Statist., 47(2):720-757, 2019. ISSN 0090-5364. URL https://doi.org/10.1214/17-AOS1633. · Zbl 1416.62259
[3] R. E. Barlow, D. J. Bartholomew, J. Bremner, and H. Brunk. Statistical inference under order restrictions: The theory and application of isotonic regression., Wiley New York, 1972. · Zbl 0246.62038
[4] D. J. Bartholomew. A test for homogeneity for ordered alternatives., Biometrika, 46:36-48, 1959a. · Zbl 0087.14202
[5] D. J. Bartholomew. A test for homogeneity for ordered alternatives ii., Biometrika, 46:328-335, 1959b. · Zbl 0090.36002
[6] H. D. Brunk. Maximum likelihood estimates of monotone parameters., Annals of Mathematical Statistics, 26:607-616, 1955. · Zbl 0066.38503
[7] H. D. Brunk. Estimation of isotonic regression. In, Nonparametric Techniques in Statistical Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969), pages 177-197. Cambridge Univ. Press, London, 1970.
[8] E. Cator. Adaptivity and optimality of the monotone least-squares estimator., Bernoulli, 17(2):714-735, 05 2011. URL https://doi.org/10.3150/10-BEJ289. · Zbl 1345.62066
[9] S. Chatterjee, A. Guntuboyina, and B. Sen. On risk bounds in isotonic and other shape restricted regression problems., The Annals of Statistics, 43 :1774-1800, 08 2015. · Zbl 1317.62032
[10] J. de Leeuw, K. Hornik, and P. Mair. Isotone optimization in r: Pool-adjacent-violators (pava) and active set methods., Journal of Statistical Software, 32:1-24, 2009.
[11] C. Durot. Sharp asymptotics for isotonic regression., Probability Theory and Related Fields, 122(2):222-240, Feb 2002. ISSN 1432-2064. URL https://doi.org/10.1007/s004400100171. · Zbl 0992.60028
[12] C. Durot, V. N. Kulikov, and H. P. Lopuhaä. The limit distribution of the \(l_\infty \)-error of grenander-type estimators., Ann. Statist., 40(3) :1578-1608, 06 2012. URL https://doi.org/10.1214/12-AOS1015. · Zbl 1257.62017
[13] C. Gao, F. Han, and C.-H. Zhang. On estimation of isotonic piecewise constant signals., arXiv preprint arXiv:1705.06386, 2017.
[14] U. Grenander. On the theory of mortality measurement: part ii., Scandinavian Actuarial Journal, 1956(2):125-153, 1956. · Zbl 0077.33715
[15] A. Guntuboyina and B. Sen. Nonparametric shape-restricted regression., Statistical Science, 33:568-594, 2018. · Zbl 1407.62135
[16] Q. Han, T. Wang, S. Chatterjee, and R. J. Samworth. Isotonic regression in general dimensions., ArXiv e-prints, 2017. · Zbl 1437.62124
[17] B. Laurent and P. Massart. Adaptive estimation of a quadratic functional by model selection., Annals of Statistics, pages 1302-1338, 2000. · Zbl 1105.62328
[18] R. E. Miles. The complete amalgamation into blocks, by weighted means, of a finite set of real numbers., Biometrika, 46:317-327, 1959. · Zbl 0090.36001
[19] T. Robertson, F. Wright, and R. Dykstra., Order Restricted Statistical Inference. Probability and Statistics Series. Wiley, 1988. ISBN 9780471917878. URL https://books.google.com/books?id=sqZfQgAACAAJ. · Zbl 0645.62028
[20] A. I. Sakhanenko. Convergence rate in the invariance principle for non-identically distributed variables with exponential moments., Advances in Probability Theory: Limit Theorems for Sums of Random Variables, pages 2-73, 1985.
[21] F. T. Wright. The asymptotic behavior of monotone regression estimates., Ann. Statist., 9(2):443-448, 1981. ISSN 0090-5364.
[22] F. Yang and R. F. Barber. Contraction and uniform convergence of isotonic regression., ArXiv e-prints, 2018. · Zbl 1478.62098
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.