×

zbMATH — the first resource for mathematics

Ordered smoothers with exponential weighting. (English) Zbl 1349.62129
Summary: The main goal in this paper is to propose a new approach to deriving oracle inequalities related to the exponential weighting method. The paper focuses on recovering an unknown vector from noisy data with the help of the family of ordered smoothers [A. Kneip, Ann. Stat. 22, No. 2, 835–866 (1994; Zbl 0815.62022)]. The estimators withing this family are aggregated using the exponential weighting method and the aim is to control the risk of the aggregated estimate. Based on the natural probabilistic properties of the unbiased risk estimate, we derive new oracle inequalities for the mean square risk and show that the exponential weighting permits to improve Kneip’s oracle inequality.

MSC:
62G08 Nonparametric regression and quantile regression
62J05 Linear regression; mixed models
62C99 Statistical decision theory
PDF BibTeX XML Cite
Full Text: DOI Euclid arXiv
References:
[1] Alquier, P. and Lounici, K. (2011). Pac-bayesian theorems for sparse regression estimation with exponential weights. Electronic Journal of Statistics 5 127-145. · Zbl 1274.62463
[2] Arias-Castro, E. and Lounici, K. Variable Selection with Exponential Weights and \(\ell_{0}\)-Penalization. · Zbl 1294.62164
[3] Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. Proc. 2nd Intern. Symp. Inf. Theory 267-281. · Zbl 0283.62006
[4] Catoni, O. (2004). Statistical Learning Theory and Stochastic Optimization . Lectures Notes in Math. 1851 . Springer-Verlag, Berlin. · Zbl 1076.93002
[5] Dalayan, A. and Salmon, J. (2012). Sharp oracle inequalities for aggregation of affine estimators. Ann. Statist. 40 2327-2355. · Zbl 1257.62038
[6] Demmler, A. and Reinsch, C. (1975). Oscillation matrices with spline smoothing. Numerische Mathematik 24 375-382. · Zbl 0297.65002
[7] Engl, H. W., Hanke, M., and Neubauer, A. (1996). Regularization of Inverse Problems . Mathematics and its Applications 375 . Kluwer Academic Publishers Group, Dordrecht. · Zbl 0859.65054
[8] Golubev, Yu. (2010). On universal oracle inequalities related to high dimensional linear models. Ann. Statist. 38 2751-2780. · Zbl 1200.62074
[9] Golubev, G. (2012). Exponential weighting and oracle inequalities for projection estimates. Problems of Information Transmission 48 269-280. · Zbl 1260.94025
[10] Green, P. J. and Silverman, B. W. (1994). Nonparametric Regression and Generalized Linear Models. A Roughness Penalty Approach . Chapman and Hall. · Zbl 0832.62032
[11] Juditsky, A. and Nemirovski, A. (2000). Functional aggregation for nonparametric regression. Ann. Statist. 28 681-712. · Zbl 1105.62338
[12] Kneip, A. (1994). Ordered linear smoothers. Ann. Statist. 22 835-866. · Zbl 0815.62022
[13] LecuĂ©, G. (2007). Simultaneous adaptation to the margin and to complexity in classification. Ann. Statist. 35 1698-1721. · Zbl 1209.62146
[14] Leung, G. and Barron, A. (2006). Information theory and mixing least-squares regressions. IEEE Transactions on Information Theory 52 3396-3410. · Zbl 1309.94051
[15] Mallows, C. L. (1973). Some comments on \(C_{p}\). Technometrics 15 661-675. · Zbl 0269.62061
[16] Nemirovski, A. (2000). Topics in Non-Parametric Statistics . Lectures Notes in Math. 1738 . Springer-Verlag, Berlin. · Zbl 0998.62033
[17] Nussbaum, M. (1985). Spline smoothing in regression models and asymptotic efficiency in \(L_{2}\). Ann. Statist. 13 984-997. · Zbl 0596.62052
[18] Rigollet, Ph. and Tsybakov, A. (2007). Linear and convex aggregation of density estimators. Math. Methods Statist. 16 260-280. · Zbl 1231.62057
[19] Rigollet, Ph. and Tsybakov, A. (2012). Sparse estimation by exponential weighting. Statist. Sci. 27 558-575. · Zbl 1331.62351
[20] Speckman, P. (1985). Spline smoothing and optimal rates of convergence in nonparametric regression. Ann. Statist. 13 970-983. · Zbl 0585.62074
[21] Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135-1151. · Zbl 0476.62035
[22] Tikhonov, A. N. and Arsenin, V. A. (1977). Solution of Ill-posed Problems . Translated from the Russian. Preface by translation editor Fritz John. Scripta Series in Mathematics. V. H. Winston & Sons, Washington, D.C.: John Wiley & Sons, New York. · Zbl 0354.65028
[23] Wahba, G. (1990). Spline Models for Observational Data . SIAM, Philadelphia. · Zbl 0813.62001
[24] Yang, Y. (2000). Combining different procedures for adaptive regression. J. Multivariate Anal. 74 135-161. · Zbl 0964.62032
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.