×

Adaptive risk bounds in unimodal regression. (English) Zbl 1441.62097

Summary: We study the statistical properties of the least squares estimator in unimodal sequence estimation. Although closely related to isotonic regression, unimodal regression has not been as extensively studied. We show that the unimodal least squares estimator is adaptive in the sense that the risk scales as a function of the number of values in the true underlying sequence. Such adaptivity properties have been shown for isotonic regression by the first author et al. [Ann. Stat. 43, No. 4, 1774–1800 (2015; Zbl 1317.62032)] and P. C. Bellec [ibid. 46, No. 2, 745–780 (2018; Zbl 1408.62066)]. A technical complication in unimodal regression is the non-convexity of the underlying parameter space. We develop a general variational representation of the risk that holds whenever the parameter space can be expressed as a finite union of convex sets, using techniques that may be of interest in other settings.

MSC:

62G08 Nonparametric regression and quantile regression
62G07 Density estimation
62G05 Nonparametric estimation
62G20 Asymptotic properties of nonparametric inference
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Amelunxen, D., Lotz, M., McCoy, M.B. and Tropp, J.A. (2014). Living on the edge: Phase transitions in convex programs with random data. Inf. Inference3 224-294. · Zbl 1339.90251
[2] Balabdaoui, F. and Jankowski, H. (2015). Maximum likelihood estimation of a unimodal probability mass function. · Zbl 1360.62152
[3] Barlow, R.E., Bartholomew, D.J., Bremner, J.M. and Brunk, H.D. (1972). Statistical Inference Under Order Restrictions. The Theory and Application of Isotonic Regression. Wiley Series in Probability and Mathematical Statistics. New York: Wiley. · Zbl 0246.62038
[4] Bellec, P.C. (2015). Sharp oracle inequalities for least squares estimators in shape restricted regression. Available at arXiv:1510.08029. · Zbl 1408.62066
[5] Bellec, P.C. (2016). Sharp oracle inequalities for Least Squares estimators in shape restricted regression. Available at arXiv:1510.08029v2. · Zbl 1408.62066
[6] Bickel, P.J. and Fan, J. (1996). Some problems on the estimation of unimodal densities. Statist. Sinica6 23-45. · Zbl 0840.62038
[7] Birgé, L. (1997). Estimation of unimodal densities without smoothness assumptions. Ann. Statist.25 970-981. · Zbl 0888.62033
[8] Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields97 113-150. · Zbl 0805.62037
[9] Boucheron, S., Lugosi, G. and Massart, P. (2013). Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford: Oxford Univ. Press. With a foreword by Michel Ledoux. · Zbl 1279.60005
[10] Boyarshinov, V. and Magdon-Ismail, M. (2006). Linear time isotonic and unimodal regression in the \(L_1\) and \(L_∞\) norms. J. Discrete Algorithms4 676-691. · Zbl 1108.62062
[11] Bro, R. and Sidiropoulos, N. (1998). Least squares algorithms under unimodality and non-negativity constraints. J. Chemom.12 223-247.
[12] Chatterjee, S. (2014). A new perspective on least squares under convex constraint. Ann. Statist.42 2340-2381. · Zbl 1302.62053
[13] Chatterjee, S., Guntuboyina, A. and Sen, B. (2015). On matrix estimation under monotonicity constraints. Available at arXiv:1506.03430. · Zbl 1419.62135
[14] Chatterjee, S., Guntuboyina, A. and Sen, B. (2015). On risk bounds in isotonic and other shape restricted regression problems. Ann. Statist.43 1774-1800. · Zbl 1317.62032
[15] Chatterjee, S. and Mukherjee, S. (2016). On estimation in tournaments and graphs under monotonicity constraints. Available at arXiv:1603.04556. · Zbl 1432.62160
[16] Eggermont, P.P.B. and LaRiccia, V.N. (2000). Maximum likelihood estimation of smooth monotone and unimodal densities. Ann. Statist.28 922-947. · Zbl 1105.62332
[17] Flammarion, N., Mao, C. and Rigollet, P. (2016). Optimal rates of statistical seriation. Available at arXiv:1607.02435v1. · Zbl 1442.62084
[18] Frisén, M. (1986). Unimodal regression. J. R. Stat. Soc. Ser. D Stat. 479-485.
[19] Gao, F. and Wellner, J.A. (2007). Entropy estimate for high-dimensional monotonic functions. J. Multivariate Anal.98 1751-1764. · Zbl 1221.62008
[20] Grotzinger, S.J. and Witzgall, C. (1984). Projections onto order simplexes. Appl. Math. Optim.12 247-270. · Zbl 0577.65049
[21] Guntuboyina, A. and Sen, B. (2015). Global risk bounds and adaptation in univariate convex regression. Probab. Theory Related Fields163 379-411. · Zbl 1327.62255
[22] Köllmann, C., Bornkamp, B. and Ickstadt, K. (2014). Unimodal regression using Bernstein-Schoenberg splines and penalties. Biometrics70 783-793. · Zbl 1393.62075
[23] Laurent, B. and Massart, P. (2000). Adaptive estimation of a quadratic functional by model selection. Ann. Statist.28 1302-1338. · Zbl 1105.62328
[24] Ledoux, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs89. Providence, RI: Amer. Math. Soc.
[25] Meyer, M. and Woodroofe, M. (2000). On the degrees of freedom in shape-restricted regression. Ann. Statist.28 1083-1104. · Zbl 1105.62340
[26] Meyer, M.C. (2001). An alternative unimodal density estimator with a consistent estimate of the mode. Statist. Sinica11 1159-1174. · Zbl 0988.62022
[27] Shah, N.B., Balakrishnan, S., Guntuboyina, A. and Wainwright, M.J. (2017). Stochastically transitive models for pairwise comparisons: Statistical and computational issues. IEEE Trans. Inform. Theory63 934-959. · Zbl 1364.94253
[28] Shoung, J.-M. and Zhang, C.-H. (2001). Least squares estimators of the mode of a unimodal regression function. Ann. Statist.29 648-665. · Zbl 1012.62044
[29] Stout, Q.F. (2000). Optimal algorithms for unimodal regression. Ann Arbor1001 48109-2122.
[30] Stout, Q.F. (2008). Unimodal regression via prefix isotonic regression. Comput. Statist. Data Anal.53 289-297. · Zbl 1231.62073
[31] van de Geer, S. (1990). Estimating a regression function. Ann. Statist.18 907-924. · Zbl 0709.62040
[32] van de Geer, S. (1993). Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Statist.21 14-44. · Zbl 0779.62033
[33] van de Geer, S.A. (2000). Applications of Empirical Process Theory. Cambridge Series in Statistical and Probabilistic Mathematics6. Cambridge: Cambridge Univ. Press. · Zbl 0953.62049
[34] Wang, Y. (1996). The \(L_2\) risk of an isotonic estimate. Comm. Statist. Theory Methods25 281-294. · Zbl 0875.62143
[35] Zhang, C.-H. (2002). Risk bounds in isotonic regression. Ann. Statist.30 528-555. · Zbl 1012.62045
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.