Adaptive prediction and estimation in linear regression with infinitely many parameters. (English) Zbl 1043.62076

Goldenshluger, A.; Tsybakov, A.

Ann. Stat. 29, No. 6, 1601-1619 (2001).

Summary: The problem of adaptive prediction and estimation in the stochastic linear regression model with infinitely many parameters is considered. We suggest a prediction method that is sharp asymptotically minimax adaptive over ellipsoids in \(\ell_2\). The method consists in an application of blockwise Stein’s rule with “weakly” geometrically increasing blocks to the penalized least squares fits of the first \(N\) coefficients. To prove the results, we develop oracle inequalities for a sequence model with correlated data.

MSC:

62M20	Inference from stochastic processes and prediction
62J05	Linear regression; mixed models
62M10	Time series, auto-correlation, regression, etc. in statistics (GARCH)
62G08	Nonparametric regression and quantile regression
62G05	Nonparametric estimation
62G20	Asymptotic properties of nonparametric inference

Keywords:

exact asymptotics of minimax risk

References:

[1]	Belitser, E. N. and Levit, B. Ya. (1995). On minimax filtering on ellipsoids. Math. Method Statist. 4 259-273. · Zbl 0836.62070
[2]	Breiman, L. and Freedman, D. (1983). How manyvariables should be entered in a regression equation? J. Amer. Statist. Assoc. 78 131-136. JSTOR: · Zbl 0513.62068
[3]	Cai, T. (1999). Adaptive wavelet estimation: a block thresholding and oracle inequalityapproach. Ann. Statist. 27 898-924. · Zbl 0954.62047
[4]	Cavalier, L. and Tsybakov, A. (2000). Sharp adaptation for inverse problems with random noise. Probab. Theory Related Fields. To appear. Available at www.proba.jussieu.fr. URL: · Zbl 1039.62031
[5]	Donoho, D. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200-1224. JSTOR: · Zbl 0869.62024
[6]	Efromovich, S. (1999). Nonparametric Curve Estimation. Springer, New York. · Zbl 0935.62039
[7]	Efroimovich, S. Yu. and Pinsker, M. S. (1984). Learning algorithm for nonparametric filtering. Automat. Remote Control, 11 1434-1440. · Zbl 0637.93069
[8]	Efromovich, S. and Pinsker, M. S. (1996). Sharp-optimal and adaptive estimation for heteroscedastic nonparametric regression. Statist. Sinica 6 925-942. · Zbl 0857.62037
[9]	Goldenshluger, A. and Tsybakov, A. (1999). Optimal prediction for linear regression with infinitelymanyparameters. Available at www.proba.jussieu.fr. URL: · Zbl 1038.62058
[10]	Johnstone, I. M. (1998). Function estimation in Gaussian noise: sequence models. Available at www-stat.stanford.edu/.
[11]	Johnstone, I. M. (1999). Wavelet shrinkage for correlated data and inverse problems: adaptivity results. Statist. Sinica 9 51-83. · Zbl 1065.62519
[12]	Lehmann, E. and Casella, G. (1998). Theory of Point Estimation. Springer, New York. · Zbl 0916.62017
[13]	Marcus, M. and Minc, H. (1992). A Survey of Matrix Theory and Matrix Inequalities. Dover, New York. · Zbl 0126.02404
[14]	Nemirovski, A. (2000). Topics in Non-Parametric Statistics. Ecole d’eté de Probabilités Saint Flour XXVIII. Lecture Notes in Math. 1738 89-277. Springer, Berlin. · Zbl 0998.62033
[15]	Petrov, V. V. (1995). Limit Theorems of Probability Theory. Clarendon Press, Oxford. · Zbl 0826.60001
[16]	Pinsker, M. S. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Problems Inform. Transmission 16 120-133. · Zbl 0452.94003
[17]	Shibata, R. (1981). An optimal selection of regression variables. Biometrika 68 45-54. JSTOR: · Zbl 0464.62054
[18]	Stein, Ch. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135-1151. · Zbl 0476.62035