zbMATH — the first resource for mathematics

On universal oracle inequalities related to high-dimensional linear models. (English) Zbl 1200.62074
Summary: This paper deals with recovering an unknown vector \(\theta \) from the noisy data \(Y = A\theta + \sigma \xi \), where \(A\) is a known \((m \times n)\)-matrix and \(\xi \) is a white Gaussian noise. It is assumed that \(n\) is large and \(A\) may be severely ill-posed. Therefore, in order to estimate \(\theta \), a spectral regularization method is used, and our goal is to choose its regularization parameter with the help of the data \(Y\). For spectral regularization methods related to the so-called ordered smoothers [see A. Kneip, Ann. Stat. 22, No. 2, 835–866 (1994; Zbl 0815.62022)], we propose new penalties in the principle of empirical risk minimization. The heuristical idea behind these penalties is related to balancing excess risks. Based on this approach, we derive a sharp oracle inequality controlling the mean square risks of data-driven spectral regularization methods.

62J05 Linear regression; mixed models
60E15 Inequalities; stochastic orderings
62H99 Multivariate analysis
15A18 Eigenvalues, singular values, and eigenvectors
62C10 Bayesian problems; characterization of Bayes procedures
62G05 Nonparametric estimation
Full Text: DOI arXiv
[1] Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Proc. 2nd Intern. Symp. Inf. Theory (P. N. Petrov and F. Csaki, eds.) 267-281. Akadémiai Kiadó, Budapest. · Zbl 0283.62006
[2] Bauer, F. and Hohage, T. (2005). A Lepskij-type stopping rule for regularized Newton methods. Inverse Problems 21 1975-1991. · Zbl 1091.65052
[3] Bissantz, N., Hohage, T., Munk, A. and Ruymgaart, F. (2007). Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM J. Numer. Anal. 45 2610-2636. · Zbl 1234.62062
[4] Cavalier, L. and Golubev, Y. (2006). Risk hull method and regularization by projections of ill-posed inverse problems. Ann. Statist. 34 1653-1677. · Zbl 1246.62082
[5] Engl, H. W., Hanke, M. and Neubauer, A. (1996). Regularization of Inverse Problems. Mathematics and Its Applications 375 . Kluwer Academic, Dordrecht. · Zbl 0859.65054
[6] Golubev, Y. (2004). The principle of penalized empirical risk in severely ill-posed problems. Probab. Theory Related Fields 130 18-38. · Zbl 1064.62011
[7] Kneip, A. (1994). Ordered linear smoothers. Ann. Statist. 22 835-866. · Zbl 0815.62022
[8] Landweber, L. (1951). An iteration formula for Fredholm integral equations of the first kind. Amer. J. Math. 73 615-624. JSTOR: · Zbl 0043.10602
[9] Loubes, J.-M. and Ludeña, C. (2008). Adaptive complexity regularization for linear inverse problems. Electron. J. Stat. 2 661-677. · Zbl 1320.62075
[10] Mair, B. A. and Ruymgaart, F. H. (1996). Statistical inverse estimation in Hilbert scale. SIAM J. Appl. Math. 56 1424-1444. JSTOR: · Zbl 0864.62020
[11] Mathé, P. (2006). The Lepskii principle revisited. Inverse Problems 22 L11-L15. · Zbl 1095.65045
[12] O’Sullivan, F. (1986). A statistical perspective on ill-posed inverse problems (with discussion). Statist. Sci. 1 502-527. · Zbl 0625.62110
[13] Pinsker, M. S. (1980). Optimal filtration of square-integrable signals in Gaussian noise. Problems Inform. Transmission 16 52-68. · Zbl 0452.94003
[14] Tikhonov, A. N. and Arsenin, V. A. (1977). Solutions of Ill-Posed Problems . Wiley, New York. · Zbl 0354.65028
[15] Van der Vaart, A. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes . Springer, New York. · Zbl 0862.60002
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.