General empirical Bayes wavelet methods and exactly adaptive minimax estimation. (English) Zbl 1064.62009

Zhang, Cun-Hui

Ann. Stat. 33, No. 1, 54-100 (2005).

Summary: In many statistical problems, stochastic signals can be represented as a sequence of noisy wavelet coefficients. We develop general empirical Bayes methods for the estimation of the true signal. Our estimators approximate certain oracle separable rules and achieve adaptation to ideal risks and exact minimax risks in broad collections of classes of signals. In particular, our estimators are uniformly adaptive to the minimum risk of separable estimators and the exact minimax risks simultaneously in Besov balls of all smoothness and shape indices, and they are uniformly superefficient in convergence rates in all compact sets in Besov spaces with a finite secondary shape parameter.
Furthermore, in classes nested between Besov balls of the same smoothness index, our estimators dominate threshold and James-Stein estimators within an infinitesimal fraction of the minimax risks. More general block empirical Bayes estimators are developed. Both white noise with drift and nonparametric regression are considered.

MSC:

62C12	Empirical decision procedures; empirical Bayes procedures
62G08	Nonparametric regression and quantile regression
42C40	Nontrigonometric harmonic analysis involving wavelets and other special systems
62G05	Nonparametric estimation
62G20	Asymptotic properties of nonparametric inference
62B10	Statistical aspects of information-theoretic topics

Keywords:

threshold estimate; white noise

Software:

EBayesThresh; ElemStatLearn

References:

[1]	Abramovich, F., Benjamini, Y., Donoho, D. L. and Johnstone, I. M. (2000). Adapting to unknown sparsity by controlling the false discovery rate. Technical Report 2000-19, Dept. Statistics, Stanford Univ. · Zbl 1092.62005
[2]	Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301–413. · Zbl 0946.62036 · doi:10.1007/s004400050210
[3]	Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis . Springer, New York. · Zbl 0572.62008
[4]	Breiman, L., Friedman, J., Olshen, R. and Stone, C. (1984). Classification and Regression Trees . Wadsworth, Belmont, CA. · Zbl 0541.62042
[5]	Brown, L. D., Cai, T. T., Low, M. and Zhang, C.-H. (2002). Asymptotic equivalence theory for nonparametric regression with random design. Ann. Statist. 30 688–707. · Zbl 1029.62044 · doi:10.1214/aos/1028674838
[6]	Brown, L. D. and Low, M. G. (1996). Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384–2398. · Zbl 0867.62022 · doi:10.1214/aos/1032181159
[7]	Brown, L. D., Low, M. G. and Zhao, L. H. (1997). Superefficiency in nonparametric function estimation. Ann. Statist. 25 2607–2625. · Zbl 0895.62043 · doi:10.1214/aos/1030741087
[8]	Cai, T. T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach. Ann. Statist. 27 898–924. · Zbl 0954.62047 · doi:10.1214/aos/1018031262
[9]	Cai, T. T. (2000). On adaptability and information-pooling in nonparametric function estimation. Technical report, Dept. Statistics, Univ. Pennsylvania. · Zbl 1148.62020
[10]	Cavalier, L. and Tsybakov, A. B. (2001). Penalized blockwise Stein’s method, monotone oracles and sharp adaptive estimation. Math. Methods Statist. 10 247–282. · Zbl 1005.62027
[11]	Cavalier, L. and Tsybakov, A. B. (2002). Sharp adaptation for inverse problems with random noise. Probab. Theory Related Fields 123 323–354. · Zbl 1039.62031 · doi:10.1007/s004400100169
[12]	Chui, C. K. (1992). An Introduction to Wavelets. Academic Press, San Diego, CA. · Zbl 0925.42016
[13]	Daubechies, I. (1992). Ten Lectures on Wavelets . SIAM, Philadelphia. · Zbl 0776.42018
[14]	Donoho, D. L. and Johnstone, I. M. (1994a). Ideal spatial adaptation via wavelet shrinkage. Biometrika 81 425–455. · Zbl 0815.62019 · doi:10.1093/biomet/81.3.425
[15]	Donoho, D. L. and Johnstone, I. M. (1994b). Minimax risk over \(\ell_p\)-balls for \(\ell_q\)-error. Probab. Theory Related Fields 99 277–303. · Zbl 0802.62006 · doi:10.1007/BF01199026
[16]	Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200–1224. · Zbl 0869.62024 · doi:10.2307/2291512
[17]	Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879–921. · Zbl 0935.62041 · doi:10.1214/aos/1024691081
[18]	Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1995). Wavelet shrinkage: Asymptopia? (with discussion). J. Roy. Statist. Soc. Ser. B 57 301–369. · Zbl 0827.62035
[19]	Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508–539. · Zbl 0860.62032 · doi:10.1214/aos/1032894451
[20]	Dubuc, S. (1986). Interpolation through an iterative scheme. J. Math. Anal. Appl. 114 185–204. · Zbl 0615.65005 · doi:10.1016/0022-247X(86)90077-6
[21]	Efromovich, S. (1985). Nonparametric estimation of a density of unknown smoothness. Theory Probab. Appl. 30 557–568. · Zbl 0593.62034 · doi:10.1137/1130067
[22]	Efromovich, S. (1999). Nonparametric Curve Estimation. Springer, New York. · Zbl 0935.62039 · doi:10.1007/b97679
[23]	Efromovich, S. and Pinsker, M. S. (1984). An adaptive algorithm for nonparametric filtering. Automat. Remote Control 11 58–65.
[24]	Efromovich, S. and Pinsker, M. S. (1986). An adaptive algorithm for minimax nonparametric estimation of a spectral density. Problems Inform. Transmission 22 62–76. · Zbl 0624.62086
[25]	Efron, B. and Morris, C. (1973). Stein’s estimation rule and its competitors—an empirical Bayes approach. J. Amer. Statist. Assoc. 68 117–130. · Zbl 0275.62005 · doi:10.2307/2284155
[26]	Foster, D. P. and George, E. I. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22 1947–1975. JSTOR: · Zbl 0829.62066 · doi:10.1214/aos/1176325766
[27]	Foster, D. P., Stine, R. A. and Wyner, A. J. (2002). Universal codes for finite sequences of integers drawn from a monotone distribution. IEEE Trans. Inform. Theory 48 1713–1720. · Zbl 1061.94019 · doi:10.1109/TIT.2002.1003851
[28]	Friedman, J. (1991). Multivariate adaptive regression splines (with discussion). Ann. Statist. 19 1–141. JSTOR: · Zbl 0765.62064 · doi:10.1214/aos/1176347963
[29]	Friedman, J. and Silverman, B. W. (1989). Flexible parsimonious smoothing and additive modeling. Technometrics 31 3–21. · Zbl 0672.65119 · doi:10.2307/1270359
[30]	Golubev, G. K. (1992). Nonparametric estimation of smooth probability densities in \(L_2\). Problems Inform. Transmission 28 44–54. · Zbl 0785.62039
[31]	Grama, I. and Nussbaum, M. (1998). Asymptotic equivalence for nonparametric generalized linear models. Probab. Theory Related Fields 111 167–214. · Zbl 0953.62039 · doi:10.1007/s004400050166
[32]	Hall, P., Kerkyacharian, G. and Picard, D. (1998). Block threshold rules for curve estimation using kernel and wavelet methods. Ann. Statist. 26 922–942. · Zbl 0929.62040 · doi:10.1214/aos/1024691082
[33]	Hall, P., Kerkyacharian, G. and Picard, D. (1999). On the minimax optimality of block thresholding wavelet estimators. Statist. Sinica 9 33–49. · Zbl 0915.62028
[34]	Hall, P. and Patil, P. (1995). Formulae for mean integrated squared error of nonlinear wavelet-based density estimators. Ann. Statist. 23 905–928. JSTOR: · Zbl 0839.62042 · doi:10.1214/aos/1176324628
[35]	Hall, P. and Patil, P. (1996). Effect of threshold rules on performance of wavelet-based curve estimators. Statist. Sinica 6 331–345. · Zbl 0843.62039
[36]	Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1998). Wavelets , Approximation , and Statistical Applications. Lecture Notes in Statist. 129 . Springer, New York. · Zbl 0899.62002
[37]	Hastie, T., Tibshirani, R. and Friedman, J. H. (2001). The Elements of Statistical Learning : Data Mining , Inference , and Prediction . Springer, New York. · Zbl 0973.62007
[38]	Ibragimov, I. A. and Khas’minskii, R. Z. (1981). Statistical Estimation : Asymptotic Theory . Springer, New York. · Zbl 0467.62026
[39]	James, W. and Stein, C. (1961). Estimation with quadratic loss. Proc. Fourth Berkeley Symp. Math. Statist. Probab. 1 361–379. Univ. California Press, Berkeley. · Zbl 1281.62026
[40]	Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1992). Estimation d’une densité de probabilité par méthode d’ondelettes. C. R. Acad. Sci. Paris Sér. I Math. 315 211–216. · Zbl 0755.62036
[41]	Johnstone, I. M. and Silverman, B. W. (1998). Empirical Bayes approaches to mixture problems and wavelet regression. Technical report, Dept. Statistics, Stanford Univ. Available at http://www-stat.stanford.edu/people/faculty/johnstone/techreports.html.
[42]	Johnstone, I. M. and Silverman, B. W. (2004). Needles and hay in haystacks: Empirical Bayes estimates of possibly sparse sequences. Ann. Statist. 32 1594–1649. · Zbl 1047.62008 · doi:10.1214/009053604000000030
[43]	Johnstone, I. M. and Silverman, B. W. (2005). Empirical Bayes selection of wavelet thresholds. Ann. Statist. 33 . · Zbl 1078.62005 · doi:10.1214/009053605000000345
[44]	Juditsky, A. (1997). Wavelet estimators: Adapting to unknown smoothness. Math. Methods Statist. 6 1–25. · Zbl 0871.62039
[45]	Lepski, O. V., Mammen, E. and Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25 929–947. · Zbl 0885.62044 · doi:10.1214/aos/1069362731
[46]	Morris, C. N. (1983). Parametric empirical Bayes inference: Theory and applications (with discussion). J. Amer. Statist. Assoc. 78 47–65. · Zbl 0506.62005 · doi:10.2307/2287098
[47]	Nussbaum, M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399–2430. · Zbl 0867.62035 · doi:10.1214/aos/1032181160
[48]	Robbins, H. (1951). Asymptotically subminimax solutions of compound statistical decision problems. Proc. Second Berkeley Symp. Math. Statist. Probab. 131–148. Univ. California Press, Berkeley. · Zbl 0044.14803
[49]	Robbins, H. (1956). An empirical Bayes approach to statistics. Proc. Third Berkeley Symp. Math. Statist. Probab. 1 157–163. Univ. California Press, Berkeley. · Zbl 0074.35302
[50]	Robbins, H. (1983). Some thoughts on empirical Bayes estimation. Ann. Statist. 11 713–723. JSTOR: · Zbl 0522.62024 · doi:10.1214/aos/1176346239
[51]	Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proc. Third Berkeley Symp. Math. Statist. Probab. 1 197–206. Univ. California Press, Berkeley. · Zbl 0073.35602
[52]	Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135–1151. JSTOR: · Zbl 0476.62035 · doi:10.1214/aos/1176345632
[53]	Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation (with discussion). Ann. Statist. 22 118–184. JSTOR: · Zbl 0827.62038 · doi:10.1214/aos/1176325361
[54]	Zhang, C.-H. (1990). Fourier methods for estimating mixing densities and distributions. Ann. Statist. 18 806–831. JSTOR: · Zbl 0778.62037 · doi:10.1214/aos/1176347627
[55]	Zhang, C.-H. (1997). Empirical Bayes and compound estimation of normal means. Statist. Sinica 7 181–193. · Zbl 0904.62008
[56]	Zhang, C.-H. (2000). General empirical Bayes wavelet methods. Technical Report 2000-007, Dept. Statistics, Rutgers Univ. Available at http://stat.rutgers.edu/ cunhui/papers.