zbMATH — the first resource for mathematics

Empirical Bayes scaling of Gaussian priors in the white noise model. (English) Zbl 1336.62039
Summary: The performance of nonparametric estimators is heavily dependent on a bandwidth parameter. In nonparametric Bayesian methods this parameter can be specified as a hyperparameter of the nonparametric prior. The value of this hyperparameter may be made dependent on the data. The empirical Bayes method is to set its value by maximizing the marginal likelihood of the data in the Bayesian framework. In this paper we analyze a particular version of this method, common in practice, that the hyperparameter scales the prior variance. We characterize the behavior of the random hyperparameter, and show that a nonparametric Bayes method using it gives optimal recovery over a scale of regularity classes. This scale is limited, however, by the regularity of the unscaled prior. While a prior can be scaled up to make it appropriate for arbitrarily rough truths, scaling cannot increase the nominal smoothness by much. Surprisingy the standard empirical Bayes method is even more limited in this respect than an oracle, deterministic scaling method. The same can be said for the hierarchical Bayes method.

62C12 Empirical decision procedures; empirical Bayes procedures
62F15 Bayesian inference
62G05 Nonparametric estimation
62G15 Nonparametric tolerance and confidence regions
62G20 Asymptotic properties of nonparametric inference
Full Text: DOI Euclid
[1] Belitser, E., and Enikeeva, F. Empirical Bayesian test of the smoothness. Math. Methods Statist. 17 , 1 (2008), 1-18. · Zbl 1282.62113
[2] Belitser, E., and Ghosal, S. Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution. Ann. Statist. 31 , 2 (2003), 536-559. Dedicated to the memory of Herbert E. Robbins. · Zbl 1039.62039
[3] Brown, L. D., and Zhao, L. H. Estimators for Gaussian models having a block-wise structure. Statist. Sinica 19 , 3 (2009), 885-903. · Zbl 1166.62050
[4] Cai, T. T., Low, M. G., and Zhao, L. H. Sharp adaptive estimation by a blockwise method. J. Nonparametr. Stat. 21 , 7 (2009), 839-850. · Zbl 1172.62024
[5] Castillo, I. Lower bounds for posterior rates with Gaussian process priors. Electron. J. Stat. 2 (2008), 1281-1299. · Zbl 1320.62067
[6] Cox, D. D. An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 , 2 (1993), 903-923. · Zbl 0778.62003
[7] Donoho, D. L. Statistical estimation and optimal recovery. Ann. Statist. 22 , 1 (1994), 238-270. · Zbl 0805.62014
[8] Donoho, D. L., Liu, R. C., and MacGibbon, B. Minimax risk over hyperrectangles, and implications. Ann. Statist. 18 , 3 (1990), 1416-1437. · Zbl 0705.62018
[9] Efroĭmovich, S. Y., and Pinsker, M. S. Estimation of square-integrable probability density of a random variable. Problemy Peredachi Informatsii 18 , 3 (1982), 19-38. · Zbl 0514.62045
[10] Efron, B., and Morris, C. Limiting the risk of Bayes and empirical Bayes estimators. II. The empirical Bayes case. J. Amer. Statist. Assoc. 67 (1972), 130-139. · Zbl 0231.62013
[11] Florens, J., and Simoni, A. Regularizing priors for linear inverse problems. · Zbl 1246.62039
[12] Freedman, D. On the Bernstein-von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 , 4 (1999), 1119-1140. · Zbl 0957.62002
[13] Ghosal, S., Ghosh, J. K., and van der Vaart, A. W. Convergence rates of posterior distributions. Ann. Statist. 28 , 2 (2000), 500-531. · Zbl 1105.62315
[14] Ghosal, S., Lember, J., and van der Vaart, A. Nonparametric Bayesian model selection and averaging. Electron. J. Stat. 2 (2008), 63-89. · Zbl 1135.62028
[15] Ghosh, J. K., and Ramamoorthi, R. V. Bayesian nonparametrics . Springer Series in Statistics. Springer-Verlag, New York, 2003.
[16] Jiang, W., and Zhang, C.-H. General maximum likelihood empirical Bayes estimation of normal means. Ann. Statist. 37 , 4 (2009), 1647-1684. · Zbl 1168.62005
[17] Johnstone, I. M., and Silverman, B. W. Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences. Ann. Statist. 32 , 4 (2004), 1594-1649. · Zbl 1047.62008
[18] Knapik, B. T., Szabó, B., van der Vaart, A. W., and van Zanten, J. H. Bayes procedures for adaptive inference in nonparametric inverse problems. Preprint, (2012). · Zbl 1334.62039
[19] Knapik, B. T., van der Vaart, A. W., and van Zanten, J. H. Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 , 5 (2011), 2626-2657. · Zbl 1232.62079
[20] Lember, J., and van der Vaart, A. On universal Bayesian adaptation. Statist. Decisions 25 , 2 (2007), 127-152. · Zbl 1146.62017
[21] Pinsker, M. S. Optimal filtration of square-integrable signals in Gaussian noise. · Zbl 0452.94003
[22] Robbins, H. An empirical Bayes approach to statistics. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954-1955, vol. I (Berkeley and Los Angeles, 1956), University of California Press, pp. 157-163.
[23] Scricciolo, C. Convergence rates for Bayesian density estimation of infinite-dimensional exponential families. Ann. Statist. 34 , 6 (2006), 2897-2920. · Zbl 1114.62043
[24] Tsybakov, A. B. Introduction à l’estimation non-paramétrique , vol. 41 of Mathématiques & Applications (Berlin) [Mathematics & Applications] . Springer-Verlag, Berlin, 2004.
[25] van der Vaart, A., and van Zanten, H. Bayesian inference with rescaled Gaussian process priors. Electron. J. Stat. 1 (2007), 433-448 (electronic). · Zbl 1140.62066
[26] van der Vaart, A. W., and van Zanten, J. H. Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. 36 , 3 (2008), 1435-1463. · Zbl 1141.60018
[27] van der Vaart, A. W., and van Zanten, J. H. Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth. Ann. Statist. 37 , 5B (2009), 2655-2675. · Zbl 1173.62021
[28] van der Vaart, A. W., and Wellner, J. A. Weak convergence and empirical processes . Springer Series in Statistics. Springer-Verlag, New York, 1996. With applications to statistics. · Zbl 0862.60002
[29] Wahba, G. Improper priors, spline smoothing and the problem of guarding against model errors in regression. J. Roy. Statist. Soc. Ser. B 40 , 3 (1978), 364-372. · Zbl 0407.62048
[30] Zhang, C.-H. General empirical Bayes wavelet methods and exactly adaptive minimax estimation. Ann. Statist. 33 , 1 (2005), 54-100. · Zbl 1064.62009
[31] Zhao, L. H. Bayesian aspects of some nonparametric problems. Ann. Statist. 28 , 2 (2000), 532-552. · Zbl 1010.62025
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.