×

Bayesian shrinkage towards sharp minimaxity. (English) Zbl 1446.62056

A very interesting question in literature is how the global shrinkage parameter, i.e., the scale parameter, in a heavy tailed prior affects the posterior contraction. The question of this work is the following: how the shape of the prior, or more specifically, the polynomial order of the prior tail affects the posterior. The answer is positive so they found out: under the sparse normal means model, the polynomial order does affect the multiplicative constant of the posterior contraction rate. More importantly, if the polynomial order is sufficiently close to 1, it will induce the optimal Bayesian posterior convergence in the sense that the Bayesian contraction rate is sharply minimax, i.e., not only the order, but also the multiplicative constant of the posterior contraction rate are optimal. This is a rather interesting statement. The above Bayesian sharp minimaxity holds when the global shrinkage parameter follows a deterministic choice which depends on the unknown sparsity \(s\). Therefore, a beta-prior modeling is further proposed, such that their sharply minimax Bayesian procedure is adaptive to the unknown \(s\). Numerical examples are given to justify the theoretical results.

MSC:

62C20 Minimax procedures in statistical decision theory
62G32 Statistics of extreme values; tail inference
60F10 Large deviations
PDF BibTeX XML Cite
Full Text: DOI arXiv Euclid

References:

[1] Armagan, A., Clyde, M. and Dunson, D. B. (2011). Generalized beta mixtures of Gaussians. In, Advances in Neural Information Processing Systems 523-531.
[2] Armagan, A., Dunson, D. B., Lee, J., Bajwa, W. U. and Strawn, N. (2013). Posterior consistency in linear models under shrinkage priors., Biometrika 100 1011-1018. · Zbl 1279.62139
[3] Bai, R. and Ghosh, M. (2017). The inverse gamma-gamma prior for optimal posterior contraction and multiple hypothesis testing., arXiv preprint arXiv:1710.04369.
[4] Barron, A. (1998). Information-theoretic characterization of Bayes performance and the choice of priors in parametric and nonparametric problems. In, Bayesian Statistics 6 (J. M. Bernardo, J. Berger, A. Dawid and A. Smith, eds.) 27-52. · Zbl 0974.62020
[5] Bhadra, A., Datta, J., Polson, N. G., Willard, B. et al. (2017). The horseshoe+ estimator of ultra-sparse signals., Bayesian Analysis 12 1105-1131. · Zbl 1384.62079
[6] Bhattacharya, A., Pati, D., Pillai, N. S. and Dunson, D. B. (2015). Dirichlet-Laplace priors for optimal shrinkage., Journal of the American Statistical Association 110 1479-1490. · Zbl 1373.62368
[7] Birgé, L. (1984). Sur un théorème de minimax et son application aux tests., Probab. Math. Statist. 3 259-282. · Zbl 0571.62036
[8] Carvalho, C. M., Polson, N. G. and Scott, J. G. (2010). The horseshoe estimator for sparse signals., Biometrika 97 465-480. · Zbl 1406.62021
[9] Castillo, I. and van der Vaart, A. (2012). Needles and straw in a haystack: Posterior concentration for possibly sparse sequences., The Annals of Statistics 40 2069-2101. · Zbl 1257.62025
[10] Donoho, D. L. and Johnstone, I. M. (1994). Minimax risk over \(l_p\)-balls for \(l_q\)-error., Probab. Theory Related Fields 277-303. · Zbl 0802.62006
[11] Donoho, D. L., Johnstone, I. M., Hoch, J. C. and Stern, A. S. (1992). Maximum entropy and the nearly black object., Journal of the Royal Statistical Society. Series B (Methodological) 41-81. · Zbl 0788.62103
[12] Efron, B. (2008). Microarrays, empirical Bayes and the two-groups model., Statistical Science 1-22. · Zbl 1327.62046
[13] Efron, B. (2012)., Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction 1. Cambridge University Press. · Zbl 1256.62007
[14] Ghosal, S., Ghosh, J. K. and Van Der Vaart, A. W. (2000). Convergence rates of posterior distributions., Annals of Statistics 28 500-531. · Zbl 1105.62315
[15] Ghosal, S. and Van Der Vaart, A. W. (2007). Convergence rates of posterior distributions for noniid observations., Annals of Statistics 35 192-223. · Zbl 1114.62060
[16] Ghosh, P. and Chakrabarti, A. (2014). Posterior concentration properties of a general class of shrinkage priors around nearly black vectors., arXiv preprint arXiv:1412.8161.
[17] Griffin, J. E. and Brown, P. J. (2011). Bayesian hyper-lassos with non-convex penalization., Australian & New Zealand Journal of Statistics 53 423-442. · Zbl 1335.62047
[18] Hans, C. (2009). Bayesian lasso regression., Biometrika 96 835-845. · Zbl 1179.62038
[19] Jiang, W. (2007). Bayesian variable selection for high dimensional generalized linear models: convergence rate of the fitted densities., Annals of Statistics 35 1487-1511. · Zbl 1123.62026
[20] Kleijn, B. J. K. and van der Vaart, A. W. (2006). Misspecification in infinite-dimensional Bayesian statistics., Annals of Statistics 34 837-877. · Zbl 1095.62031
[21] Laurent, B. and Massart, P. (2000). Adaptive estimation of a quadratic functional by model selection., Annals of Statistics 1302-1338. · Zbl 1105.62328
[22] Le Cam, L. (1986)., Asymptotic Methods in Statistical Decision Theory. Springer, New York. · Zbl 0605.62002
[23] Park, T. and Casella, G. (2008). The Bayesian lasso., Journal of the American Statistical Association 103 681-686. · Zbl 1330.62292
[24] Rocková, V. (2015). Bayesian estimation of sparse signals with a continuous spike-and-slab prior. Submitted manuscript, 1-34.
[25] Singh, D., Febbo, P. G., Ross, K., Jackson, D. G., Manola, J., Ladd, C., Tamayo, P., Renshaw, A. A., D’Amico, A. V., Richie, J. P. et al. (2002). Gene expression correlates of clinical prostate cancer behavior., Cancer Cell 1 203-209.
[26] Song, Q. and Liang, F. (2017). Nearly optimal Bayesian shrinkage for high dimensional regression., arXiv:1712.08964.
[27] van der Pas, S. L., Kleijn, B. J. K. and van der Vaart A. W. (2014). The horseshoe estimator: Posterior concentration around nearly black vectors., Electronic Journal of Statistics 2 2585-2618. · Zbl 1309.62060
[28] van der Pas, S., Salomond, J.-B. and Schmidt-Hieber, J. (2016). Conditions for posterior contraction in the sparse normal means problem., Electronic Journal of Statistics 10 976-1000. · Zbl 1343.62012
[29] van der Pas, S. L., Szabo, B. and van der Vaart, A. (2017). Adaptive posterior contraction rates for the horseshoe., arXiv:1702.03698. · Zbl 1373.62140
[30] van der Pas, S., Szabó, B. and van der Vaart, A. (2017). Uncertainty quantification for the horseshoe (with discussion)., Bayesian Analysis 12 1221-1274. · Zbl 1384.62155
[31] Zhang, C.-H. (2012). Minimax \(L_q\) risk in \(L_p\) balls. In, Contemporary Developments in Bayesian Analysis and Statistical Decision Theory: A Festschrift for William E. Strawderman 78-89. Institute of Mathematical Statistics.
[32] Zubkov, A. · Zbl 1280.60016
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.