Minimax risk over \(l_ p\)-balls for \(l_ q\)-error. (English) Zbl 0802.62006

Summary: Consider estimating the mean vector \(\theta\) from data \(N_ n (\theta, \sigma^ 2 I)\) with \(l_ q\) norm loss, \(q\geq 1\), when \(\theta\) is known to lie in an \(n\)-dimensional \(l_ p\) ball, \(p\in (0,\infty)\). For large \(n\), the ratio of minimax linear risk to minimax risk can be arbitrarily large if \(p<q\). Obvious exceptions aside, the limiting ratio equals 1 only if \(p=q =2\).
Our arguments are mostly indirect, involving a reduction to a univariate Bayes minimax problem. When \(p<q\), simple nonlinear coordinatewise threshold rules are asymptotically minimax at small signal-to-noise ratios, and within a bounded factor of asymptotic minimaxity in general. We also give asymptotic evaluations of the minimax linear risk. Our results are basic to a theory of estimation in Besov spaces using wavelet bases (to appear elsewhere).


62C20 Minimax procedures in statistical decision theory
62F12 Asymptotic properties of parametric estimators
62G20 Asymptotic properties of nonparametric inference
Full Text: DOI


[1] Bickel, P.J.: Minimax estimation of the mean of a normal distribution when the parameter space is restricted. Ann. Stat.9, 1301-1309 (1981) · Zbl 0484.62013
[2] Birgé, L., Massart, P.: Rates of convergence for minimum contrast estimators. Technical Report Université Paris VI. Probab. Theory Relat. Fields97, 113-150 (1993) · Zbl 0805.62037
[3] Brown, L.D.: Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Stat.42, 855-903 (1971) · Zbl 0246.62016
[4] Brown, L.D., Johnstone, I.M., MacGibbon, K.B.: Variation Diminishing Transformations: A direct approach to total positivity and its statistical applications. J. Am. Stat. Assoc.76, 824-832 (1981) · Zbl 0481.62021
[5] Casella, G., Strawderman, W.E.: Estimating a bounded normal mean. Ann. Stat.9, 870-878 (1981) · Zbl 0474.62010
[6] Daubechies, I.: Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math.41, 909-996 (1988) · Zbl 0644.42026
[7] Daubechies, I.: Ten Lectures on Wavelets SIAM: Philadelphia (1992) · Zbl 0776.42018
[8] Donoho, D.L.: Gelfandn-widths and the method of least squares. (Preprint, 1990)
[9] Donoho, D.L.: Asymptotic minimax risk for sup-norm loss: solution via optimal recovery. Probab. Theory Relat. Fields (to appear, 1994) · Zbl 0802.62007
[10] Donoho, D.L., Johnstone, I.M.: Minimax risk overl p -balls. (Technical Report No. 322) Department of Statistics, Stanford: Stanford University 1989
[11] Donoho, D.L., Johnstone, I.M.: Minimax risk overl p -balls forl q -error (Technical Report) Department of Statistics. Stanford: Stanford University 1992
[12] Donoho, D.L., Johnstone, I.M.: Minimax estimation via wavelet shrinkage. (Technical Report). Ann. Stat. (to appear, 1995a) · Zbl 0935.62041
[13] Donoho, D.L., Johnstone, I.M.: Adapting to unknown smoothness via Wavelet shrinkage. J. Am. Stat. Assoc. (to appear, 1995b) · Zbl 0869.62024
[14] Donoho, D.L., Johnstone, I.M.: Non-classical Minimax Theorems, Thresholding, and Adaptation. Manuscript (1993)
[15] Donoho, D.L., Johnstone, I.M.: Ideal Spatial Adaptation via Wavelet Shrinkage. Biometrika (to appear, 1994) · Zbl 0815.62019
[16] Donoho, D.L., Johnstone, I.M., Hoch, J.C., Stern, A.S.: Maximum Entropy and the Nearly Black Image. J. R. Stat. Soc. Ser. B54, 41-81 (1992) with discussion · Zbl 0788.62103
[17] Donoho, D.L., Johnstone, I.M., Kerkyacharian, G., Picard, D.: Wavelet Shrinkage: Asymptopia? · Zbl 0827.62035
[18] Donoho, D.J., Johnstone, I.M., Kerkyacharian, G., Picard, D.: Density estimation by wavelet thresholding. (Technical Report) Department of Statistics. Stanford: Stanford University 1993 · Zbl 0860.62032
[19] Donoho, D.L., Liu, R.C., MacGibbon, K.B.: Minimax risk over hyperrectangles, and implications. Ann. Stat.18, 1416-1437 (1990) · Zbl 0705.62018
[20] Donoho, D.L., Liu, R.C.: Geometrizing Rates of Convergence, III. Ann. Stat.19, 668-701 (1991) · Zbl 0754.62029
[21] Feldman, I.: Constrained minimax estimation of the mean of the normal distribution with known variance. Ann. Stat.19, 2259-2265 (1991) · Zbl 0745.62021
[22] Huber, P.J.: Robust Estimation of a Location Parameter. Ann. Math. Stat.35, 73-101 (1964) · Zbl 0136.39805
[23] Huber, P.J.: Fisher Information and Spline Interpolation. Ann. Stat.2, 1029-1034 (1974) · Zbl 0289.62032
[24] Ibragimov, I.A., Khasminskii, R.Z.: Nonparametric estimation of the value of a linear functional in a Gaussian white noise. Theory Probab. Appl.29, 1-32 (1984) · Zbl 0532.62061
[25] Ibragimov, I.A., Hasminskii, R.Z.: On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Stat.18, 999-1010 (1990) · Zbl 0705.62039
[26] Johnstone, I.M.: A moment inequality forL q estimation. Stat. Probab. Lett.12, 289-290 (1991) · Zbl 0735.62034
[27] Johnstone, I.M.: Minimax Bayes, asymptotic minimax and sparse wavelet priors. Statistical Decision Theory and Related Topics V (pp. 303-326). Berlin Heidelberg New York: Springer 1994
[28] Karlin, S.: Total Positivity. Stanford: Stanford University Press 1968
[29] Korostelev, A.P.: Asymptotic minimax estimation of regression function in the uniform norm. Teor. Veoryatn. Priment.37 (in russian). Theory Probab. Appl.37 (1993) (in english)
[30] LeCam, L.: Asymptotic methods in statistical decision theory. Berlin Heidelberg New York: Springer 1986
[31] Lehmann, E.L.: Theory of Point Estimation. New York: Wiley 1983 · Zbl 0522.62020
[32] Lindenstrauss, J., Tzafiri, L.: The Classical Banach Spaces, II. Berlin Heidelberg New York: Springer 1979
[33] Meyer, Y.: Ondelettes. Hermann: Paris 1990; Cambridge University Press 1993 (English translation)
[34] Nemirovskii, A.S., Polyak, B.T., Tsybakov, A.B.: Rate of convergence of non-parametric estimates of maximum-likelihood type. Prob. Inf. Transm.21, 258-272 (1985) · Zbl 0616.62048
[35] Pinsker, M.S.: Optimal Filtration of square-integrable signals in Gaussian White Noise. Prob. Inf. Transm.16, 120-133 (1980) · Zbl 0452.94003
[36] Rockafellar, R.T.: Convex Analysis. Princeton: Princeton University Press 1970 · Zbl 0193.18401
[37] Sacks, J., Strawderman, W.E.: Improving on linear minimax estimates. (Stat. Decis. Theory Relat. Topics III, vol. 2 pp. 287-304) J.R. Stat. Soc., Ser. B (with discussion) (to appear, 1995) Berlin Heidelberg New York: Springer 1982
[38] Sion, M.: On general minimax theorems. Pac. J. Math.8, 171-176 (1958) · Zbl 0081.11502
[39] Van de Geer, S.: Estimating a regression function. Ann. Stat.18, 907-924 (1990) · Zbl 0709.62040
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.