Parameter tuning in pointwise adaptation using a propagation approach. (English) Zbl 1173.62028

Summary: This paper discusses the problem of adaptive estimation of a univariate object like the value of a regression function at a given point or a linear functional in a linear inverse problem. We consider an adaptive procedure originated by O. V. Lepskij [Theory Probab. Appl. 35, No. 3, 454–466 (1990); translation from Teor. Veroyatn. Primen. 35, No. 3, 459–470 (1990; Zbl 0725.62075)] that selects in a data-driven way one estimate out of a given class of estimates ordered by their variability. A serious problem with using this and similar procedures is the choice of some tuning parameters like thresholds. Numerical results show that the theoretically recommended proposals appear to be too conservative and lead to a strong oversmoothing effect. A careful choice of the parameters of the procedure is extremely important for getting a reasonable quality of estimation.
The main contribution of this paper is a new approach for choosing the parameters of the procedure by providing the prescribed behavior of the resulting estimate in a simple parametric situation. We establish a non-asymptotical “oracle” bound, which shows that the estimation risk is, up to a logarithmic multiplier, equal to the risk of the “oracle” estimate that is optimally selected from the given family. A numerical study demonstrates a good performance of the resulting procedure in a number of simulated examples.


62G08 Nonparametric regression and quantile regression
62G05 Nonparametric estimation
62G10 Nonparametric hypothesis testing
65C60 Computational problems in statistics (MSC2010)
60E15 Inequalities; stochastic orderings


Zbl 0725.62075
Full Text: DOI arXiv


[1] Bauer, F. (2007). Some considerations concerning regularization and parameter choice algorithms. Inverse Problems 23 837-858. · Zbl 1109.62020
[2] Birgé, L. and Massart, P. (1993). Rate of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113-150. · Zbl 0805.62037
[3] Birgé, L. and Massart, P. (1998). Minimum contrast estimators on sieves: Exponential bounds and rates of convergence. Bernoulli 4 329-375. · Zbl 0954.62033
[4] Birgé, L. (2006). Model selection via testing: an alternative to (penalized) maximum likelihood estimators. Ann. Inst. H. Poincaré Probab. Statist. 42 273-325. · Zbl 1333.62094
[5] Cai, T. T. and Hall, P. (2006). Prediction in functional linear regression Ann. Statist. 34 2159-2179. · Zbl 1106.62036
[6] Cavalier, L. (2001). On the problem of local adaptive estimation in tomography. Bernoulli 7 63-78. · Zbl 0966.62021
[7] Goldenshluger, A. (1998). Nonparametric estimation of transfer functions: Rates of convergence and adaptation. IEEE Trans. Inform. Theory 44 644-658. · Zbl 0902.62043
[8] Goldenshluger, A. (1999). On pointwise adaptive nonparametric deconvolution. Bernoulli 5 907-925. · Zbl 0953.62033
[9] Goldenshluger, A. and Pereverzev, S. (2000). Adaptive estimation of linear functionals in Hilbert scales from indirect white noise observations. Probab. Theory Related Fields 118 169-186. · Zbl 1055.62523
[10] Goldenshluger, A. and Pereverzev, S. (2003). On adaptive inverse estimation of linear functionals in Hilbert scales. Bernoulli 9 783-807. · Zbl 1055.62034
[11] Golubev, G. K. (2004). The Method of Risk Envelope in Estimation of Linear Functionals. Problems Inform. Transmission 40 53-65. Translated from Problemy Peredachi Informatsii 1 (2004) 58-72. · Zbl 1169.62318
[12] Juditsky, A., Rigollet, P. and Tsybakov, A. B. (2008). Learning by mirror averaging. Ann. Statist. 36 2183-2206. · Zbl 1274.62288
[13] Lepskii, O. V. (1990). A problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35 454-466. Translated from Teor. Veroyatnost. i Primenen. 35 (1990) 459-470. · Zbl 0725.62075
[14] Lepskii, O. V. (1991). Asymptotically minimax adaptive estimation. I. Upper Bounds. Optimally adaptive estimates. Theory Probab. Appl. 36 682-697. · Zbl 0776.62039
[15] Lepskii, O. V. (1992). Asymptotically minimax adaptive estimation. II. Schemes without optimal adaptation: Adaptive estimators. Theory Probab. Appl. 37 433-448. · Zbl 0787.62087
[16] Lepski, O. V., Mammen, E. and Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25 929-947. · Zbl 0885.62044
[17] Lepski, O. V. and Spokoiny, V. (1997). Optimal pointwise adaptive methods in nonparametric estimation. Ann. Statist. 25 2512-2546. · Zbl 0894.62041
[18] Spokoiny, V. (2009). Local Parametric Estimation . Springer, New York. · Zbl 1160.62084
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.