×
van der Pas, S. L.; Kleijn, B. J. K.; van der Vaart, A. W.

The horseshoe estimator: posterior concentration around nearly black vectors. (English) Zbl 1309.62060

Electron. J. Stat. 8, No. 2, 2585-2618 (2014).
Summary: We consider the horseshoe estimator due to [C. M. Carvalho et al., Biometrika 97, No. 2, 465–480 (2010; Zbl 1406.62021)] for the multivariate normal mean model in the situation that the mean vector is sparse in the nearly black sense. We assume the frequentist framework where the data is generated according to a fixed mean vector. We show that if the number of nonzero parameters of the mean vector is known, the horseshoe estimator attains the minimax \(\ell_{2}\) risk, possibly up to a multiplicative constant. We provide conditions under which the horseshoe estimator combined with an empirical Bayes estimate of the number of nonzero means still yields the minimax risk. We furthermore prove an upper bound on the rate of contraction of the posterior distribution around the horseshoe estimator, and a lower bound on the posterior variance. These bounds indicate that the posterior distribution of the horseshoe prior may be more informative than that of other one-component priors, including the Lasso.

Cited in 2 Reviews
Cited in 62 Documents

MSC:

62F15 Bayesian inference
62F10 Point estimation

Keywords:

sparsity; horseshoe prior; worst case risk; Bayesian inference; empirical Bayes; posterior contraction; normal means model

Citations:

Zbl 1406.62021

Software:

EBayesThresh
Cite Review PDF
Full Text: DOI arXiv Euclid

References:

[1] Armagan, A., Dunson, D. B. and Lee, J. (2013). Generalized Double Pareto Shrinkage., Statistica Sinica 23 119-143. · Zbl 1259.62061 · doi:10.5705/ss.2011.048
[2] Bhattacharya, A., Pati, D., Pillai, N. S. and Dunson, D. B. (2012). Bayesian Shrinkage.,
[3] Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous Analysis of Lasso and Dantzig Selector., The Annals of Statistics 37 1705-1732. · Zbl 1173.62022 · doi:10.1214/08-AOS620
[4] Bogdan, M., Ghosh, J. K. and Tokdar, S. T. (2008). A Comparison of the Benjamini-Hochberg Procedure with Some Bayesian Rules for Multiple Testing. In, Beyond Parametrics in Interdisciplinary Research: Festschrift in Honor of Professor Pranab K. Sen. The Institute of Mathematical Statistics. · doi:10.1214/193940307000000158
[5] Carvalho, C. M., Polson, N. G. and Scott, J. G. (2009). Handling Sparsity via the Horseshoe., Journal of Machine Learning Research, W&CP 5 73-80.
[6] Carvalho, C. M., Polson, N. G. and Scott, J. G. (2010). The Horseshoe Estimator for Sparse Signals., Biometrika 97 465-480. · Zbl 1406.62021 · doi:10.1093/biomet/asq017
[7] Castillo, I., Schmidt-Hieber, J. and Van der Vaart, A. W. (2014). Bayesian Linear Regression with Sparse Priors., · Zbl 1486.62197
[8] Castillo, I. and Van der Vaart, A. W. (2012). Needles and Straw in a Haystack: Posterior Concentration for Possibly Sparse Sequences., The Annals of Statistics 40 2069-2101. · Zbl 1257.62025 · doi:10.1214/12-AOS1029
[9] Chernoff, H. (1952). A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the Sum of Observations., The Annals of Mathematical Statistics 23 493-507. · Zbl 0048.11804 · doi:10.1214/aoms/1177729330
[10] Datta, J. and Ghosh, J. K. (2013). Asymptotic Properties of Bayes Risk for the Horseshoe Prior., Bayesian Analysis 8 111-132. · Zbl 1329.62122 · doi:10.1214/13-BA805
[11] Donoho, D. L., Johnstone, I. M., Hoch, J. C. and Stern, A. S. (1992). Maximum Entropy and the Nearly Black Object (with discussion)., Journal of the Royal Statistical Society. Series B (Methodological) 54 41-81. · Zbl 0788.62103
[12] Efron, B. (2008). Microarrays, Empirical Bayes and the Two-Groups Model., Statistical Science 23 1-22. · Zbl 1327.62046 · doi:10.1214/07-STS236
[13] Ghosal, S., Ghosh, J. K. and Van der Vaart, A. W. (2000). Convergence Rates of Posterior Distributions., The Annals of Statistics 28 500-531. · Zbl 1105.62315 · doi:10.1214/aos/1016218228
[14] Gradshteyn, I. S. and Ryzhik, I. M. (1965)., Table of Integrals, Series and Products . Academic Press. · Zbl 0918.65002
[15] Griffin, J. E. and Brown, P. J. (2010). Inference with Normal-Gamma Prior Distributions in Regression Problems., Bayesian Analysis 5 171-188. · Zbl 1330.62128 · doi:10.1214/10-BA507
[16] Jiang, W. and Zhang, C.-H. (2009). General Maximum Likelihood Empirical Bayes Estimation of Normal Means., The Annals of Statistics 37 1647-1684. · Zbl 1168.62005 · doi:10.1214/08-AOS638
[17] Johnstone, I. M. and Silverman, B. W. (2004). Needles and Straw in Haystacks: Empirical Bayes Estimates of Possibly Sparse Sequences., The Annals of Statistics 32 1594-1649. · Zbl 1047.62008 · doi:10.1214/009053604000000030
[18] Koenker, R. (2014). A Gaussian Compound Decision Bakeoff., Stat. 3 12-16.
[19] Koenker, R. and Mizera, I. (2014). Convex Optimization, Shape Constraints, Compound Decisions and Empirical Bayes Rules., Journal of the American Statistical Association 109 674-685. · Zbl 1367.62020 · doi:10.1080/01621459.2013.869224
[20] Martin, R. and Walker, S. G. (2014). Asymptotically Minimax Empirical Bayes Estimation of a Sparse Normal Mean Vector., Electronic Journal of Statistics 8 2188-2206. · Zbl 1302.62015 · doi:10.1214/14-EJS949
[21] Miller, P. D. (2006)., Applied Asymptotic Analysis . Graduate Studies in Mathematics 75 . The American Mathematical Society. · Zbl 1101.41031
[22] Mitchell, T. J. and Beauchamp, J. J. (1988). Bayesian Variable Selection in Linear Regression., Journal of the American Statistical Association 83 1023-1032. · Zbl 0673.62051 · doi:10.2307/2290129
[23] Pericchi, L. R. and Smith, A. F. M. (1992). Exact and Approximate Posterior Moments for a Normal Location Parameter., Journal of the Royal Statistical Society. Series B (Methodological) 54 793-804. · Zbl 0775.62024
[24] Polson, N. G. and Scott, J. G. (2010). Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction. In, Bayesian Statistics 9 (J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West, eds.) Oxford University Press. · doi:10.1093/acprof:oso/9780199694587.003.0017
[25] Polson, N. G. and Scott, J. G. (2012a). Good, Great or Lucky? Screening for Firms with Sustained Superior Performance Using Heavy-Tailed Priors., The Annals of Applied Statistics 6 161-185. · Zbl 1235.91144 · doi:10.1214/11-AOAS512
[26] Polson, N. G. and Scott, J. G. (2012b). On the Half-Cauchy Prior for a Global Scale Parameter., Bayesian Analysis 7 887-902. · Zbl 1330.62148 · doi:10.1214/12-BA730
[27] Scott, J. G. (2010). Parameter Expansion in Local-Shrinkage Models.,
[28] Scott, J. G. (2011). Bayesian Estimation of Intensity Surfaces on the Sphere via Needlet Shrinkage and Selection., Bayesian Analysis 6 307-328. · Zbl 1330.62221 · doi:10.1214/11-BA611
[29] Scott, J. G. and Berger, J. O. (2010). Bayes and Empirical-Bayes Multiplicity Adjustment in the Variable-Selection Problem., The Annals of Statistics 38 2587-2619. · Zbl 1200.62020 · doi:10.1214/10-AOS792
[30] Shafer, R. E. (1966). Elementary Problems. Problem E 1867., The American Mathematical Monthly 73 309.
[31] Tibshirani, R. (1996). Regression Shrinkage and Selection via the Lasso., Journal of the Royal Statistical Society. Series B (Methodological) 58 267-288. · Zbl 0850.62538
[32] Yuan, M. and Lin, Y. (2005). Efficient Empirical Bayes Variable Selection and Estimation in Linear Models., Journal of the American Statistical Association 100 1215-1225. · Zbl 1117.62453 · doi:10.1198/016214505000000367
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.