zbMATH — the first resource for mathematics

Bayesian aspects of some nonparametric problems. (English) Zbl 1010.62025
Summary: We study a Bayesian approach to nonparametric function estimation problems such as nonparametric regression and signal estimation. We consider the asymptotic properties of Bayes procedures for conjugate (= Gaussian) priors. We show that so long as the prior puts nonzero measure on the very large parameter set of interest then the Bayes estimators are not satisfactory. More specifically, we show that these estimators do not achieve the correct minimax rate over norm bounded sets in the parameter space. Thus all Bayes estimators for proper Gaussian priors have zero asymptotic efficiency in this minimax sense. We then present a class of priors whose Bayes procedures attain the optimal minimax rate of convergence. These priors may be viewed as compound, or hierarchical, mixtures of suitable Gaussian distributions.

62F15 Bayesian inference
62G20 Asymptotic properties of nonparametric inference
Full Text: DOI Euclid
[1] Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York. · Zbl 0572.62008
[2] Brown, L. D. (1986). Fundamentals of Statistical Exponential Families with Applications in Statistical Decision Theory. IMS, Hayward, CA. · Zbl 0685.62002
[3] Brown, L. D. and Low, M. G. (1996). Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384-2398. · Zbl 0867.62022
[4] Cox, D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903-923. · Zbl 0778.62003
[5] Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1-67. · Zbl 0595.62022
[6] Diaconis, P. and Freedman, D. (1998). On the Bernstein-von Mises theorem with infinite dimensional parameters. Technical Report 492, Dept. Statistics, Univ. California, Berkeley Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. Ann. Statist. 7 269-281.
[7] Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879-921. · Zbl 0935.62041
[8] Donoho, D. L. and Liu, R. (1991). Geometrizing rates of convergence III. Ann. Statist. 19 668- 701. · Zbl 0754.62029
[9] Donoho, D. L., Liu, R. and MacGibbon, B. (1990). Minimax risk over hyperrectangles and implications. Ann. Statist. 18 1416-1437. · Zbl 0705.62018
[10] Donoho, D. L. and Low, M. G. (1992). Renormalization exponents and optimal pointwise rates of convergence. Ann. Statist. 20 944-970. · Zbl 0797.62032
[11] Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209-230. · Zbl 0255.62037
[12] Freedman, D. (1999). On the Bernstein-von Mises theorem with infinite dimensional parameters. Ann. Statist. 27 1119-1140. · Zbl 0957.62002
[13] Ibragimov, I. A. and Hasminskii, R. Z. (1977). Estimation of infinite-dimensional parameter in white Gaussian noise. Dokl. Akad. Nauk SSSR 236 1053-1056. · Zbl 0389.62023
[14] Klemelä, J. and Nussbaum, M. (1998). Constructive asymptotic equivalence of density estimation and Gaussian white noise. Unpublished manuscript.
[15] Kuo, H. (1975). Gaussian Measures in Banach Spaces. Lecture Notes in Math. 403. Springer, New York. · Zbl 0306.28010
[16] LeCam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York. · Zbl 0605.62002
[17] Mandelbaum, A. (1983). Linear estimators of the mean of a Gaussian distribution on a Hilbert space. Ph. D. dissertation, Cornell Univ. · Zbl 0542.57015
[18] Mandelbaum, A. (1984). All admissible linear estimators of the mean of a Gaussian distribution on a Hilbert space. Ann. Statist. 12 1448-1466. · Zbl 0558.62009
[19] Nussbaum, M. (1996). Asymptotic equivalence of density estimation and white noise. Ann. Statist. 24 2399-2430. · Zbl 0867.62035
[20] Pinsker, M. S. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Problemy Peredachi Informatsii 16 52-68 (in Russian). (Translation in Problems Imform. Transmission 16 120-133.) · Zbl 0452.94003
[21] Shen, X. and Wasserman, L. (1998). Rates of convergence of posterior distributions. Unpublished manuscript. · Zbl 1041.62022
[22] Van der Linde, A. (1993). A note on smoothing splines as Bayesian estimates. Statist. Decisions 11 61-67. · Zbl 0798.62038
[23] Van der Linde, A. (1995). Splines from a Bayesian point of view. Test 4 63-81. · Zbl 0839.62044
[24] Wahba, G. (1978). Improper priors, spline smoothing and the problem of guarding against model errors in regression. J. Roy. Statist. Soc. Ser. B 40 364-372. · Zbl 0407.62048
[25] Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia. · Zbl 0813.62001
[26] Zhao, L. H. (1993). Frequentist and Bayesian Aspects of some nonparametric estimation problems. Ph.D. dissertation, Cornell Univ.
[27] Zhao, L. H. (1996). Bayesian aspects of some nonparametric estimation problems (abstract). IMS Bulletin 25 20.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.