×

Convergence rates of posterior distributions for non iid observations. (English) Zbl 1114.62060

Summary: We consider the asymptotic behavior of posterior distributions and Bayes estimators based on observations which are required to be neither independent nor identically distributed. We give general results on the rate of convergence of the posterior measure relative to distances derived from a testing criterion. We then specialize our results to independent, nonidentically distributed observations, Markov processes, stationary Gaussian time series and the white noise model. We apply our general results to several examples of infinite-dimensional statistical models including nonparametric regression with normal errors, binary regression, Poisson regression, an interval censoring model, Whittle estimation of the spectral density of a time series and a nonlinear autoregressive model.

MSC:

62G20 Asymptotic properties of nonparametric inference
62F15 Bayesian inference
62B15 Theory of statistical experiments
62G08 Nonparametric regression and quantile regression
62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62M99 Inference from stochastic processes

References:

[1] Amewou-Atisso, M., Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (2003). Posterior consistency for semiparametric regression problems. Bernoulli 9 291–312. · Zbl 1015.62018 · doi:10.3150/bj/1068128979
[2] Barron, A., Schervish, M. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536–561. · Zbl 0980.62039 · doi:10.1214/aos/1018031206
[3] Birgé, L. (1983). Approximation dans les espaces métriques et théorie de l’estimation. Z. Wahrsch. Verw. Gebiete 65 181–237. · Zbl 0506.62026 · doi:10.1007/BF00532480
[4] Birgé, L. (1983). Robust testing for independent non-identically distributed variables and Markov chains. In Specifying Statistical Models. From Parametric to Non-Parametric. Using Bayesian or Non-Bayesian Approaches . Lecture Notes in Statist. 16 134–162. Springer, New York. · Zbl 0509.62036
[5] Birgé, L. (2006). Model selection via testing: An alternative to (penalized) maximum likelihood estimators. Ann. Inst. H. Poincaré Probab. Statist. 42 273–325 · Zbl 1333.62094 · doi:10.1016/j.anihpb.2005.04.004
[6] Brockwell, P. J. and Davis, R. A. (1991). Time Series : Theory and Methods , 2nd ed. Springer, New York. · Zbl 0709.62080
[7] Choudhuri, N., Ghosal, S. and Roy, A. (2004). Bayesian estimation of the spectral density of a time series. J. Amer. Statist. Assoc. 99 1050–1059. · Zbl 1055.62100 · doi:10.1198/016214504000000557
[8] Choudhuri, N., Ghosal, S. and Roy, A. (2004). Contiguity of the Whittle measure for a Gaussian time series. Biometrika 91 211–218. · Zbl 1132.62350 · doi:10.1093/biomet/91.1.211
[9] Chow, Y. S. and Teicher, H. (1978). Probability Theory. Independence, Interchangeability, Martingales . Springer, New York. · Zbl 0399.60001
[10] Dahlhaus, R. (1988). Empirical spectral processes and their applications to time series analysis. Stochastic Process. Appl. 30 69–83. · Zbl 0655.60033 · doi:10.1016/0304-4149(88)90076-2
[11] de Boor, C. (1978). A Practical Guide to Splines . Springer, New York. · Zbl 0406.41003
[12] Ghosal, S. (2001). Convergence rates for density estimation with Bernstein polynomials. Ann. Statist. 29 1264–1280. · Zbl 1043.62024 · doi:10.1214/aos/1013203453
[13] Ghosal, S., Ghosh, J. K. and Samanta, T. (1995). On convergence of posterior distributions. Ann. Statist. 23 2145–2152. · Zbl 0858.62024 · doi:10.1214/aos/1034713651
[14] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531. · Zbl 1105.62315 · doi:10.1214/aos/1016218228
[15] Ghosal, S., Lember, J. and van der Vaart, A. W. (2003). On Bayesian adaptation. Acta Appl. Math. 79 165–175. · Zbl 1030.62030 · doi:10.1023/A:1025856016236
[16] Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29 1233–1263. · Zbl 1043.62025 · doi:10.1214/aos/1013203452
[17] Ghosal, S. and van der Vaart, A. W. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities. Ann. Statist. 35 . To appear. Available at www4.stat.ncsu.edu/ sghosal/papers.html. · Zbl 1117.62046 · doi:10.1214/009053606000001271
[18] Ibragimov, I. A. (1962). Some limit theorems for stationary processes. Theory Probab. Appl. 7 349–382. · Zbl 0119.14204 · doi:10.1137/1107036
[19] Ibragimov, I. A. and Has’minskii, R. Z. (1981). Statistical Estimation: Asymptotic Theory . Springer, New York. · Zbl 0467.62026
[20] Le Cam, L. M. (1973). Convergence of estimates under dimensionality restrictions. Ann. Statist. 1 38–53. · Zbl 0255.62006 · doi:10.1214/aos/1193342380
[21] Le Cam, L. M. (1975). On local and global properties in the theory of asymptotic normality of experiments. In Stochastic Processes and Related Topics (M. L. Puri, ed.) 13–54. Academic Press, New York. · Zbl 0389.62011
[22] Le Cam, L. M. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York. · Zbl 0605.62002
[23] Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, New York. · Zbl 0925.60001
[24] Petrone, S. (1999). Bayesian density estimation using Bernstein polynomials. Canad. J. Statist. 27 105–126. JSTOR: · Zbl 0929.62044 · doi:10.2307/3315494
[25] Pollard, D. (1990). Empirical Processes: Theory and Applications. IMS, Hayward, CA. · Zbl 0741.60001
[26] Shen, X. (2002). Asymptotic normality of semiparametric and nonparametric posterior distributions. J. Amer. Statist. Assoc. 97 222–235. JSTOR: · Zbl 1073.62517 · doi:10.1198/016214502753479365
[27] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714. · Zbl 1041.62022 · doi:10.1214/aos/1009210686
[28] Stone, C. J. (1990). Large-sample inference for log-spline models. Ann. Statist. 18 717–741. · Zbl 0712.62036 · doi:10.1214/aos/1176347622
[29] Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation (with discussion). Ann. Statist. 22 118–184. · Zbl 0827.62038 · doi:10.1214/aos/1176325361
[30] van der Meulen, F., van der Vaart, A. W. and van Zanten, J. H. (2006). Convergence rates of posterior distributions for Brownian semimartingale models. Bernoulli 12 863–888. · Zbl 1142.62057 · doi:10.3150/bj/1161614950
[31] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes . With Applications to Statistics . Springer, New York. · Zbl 0862.60002
[32] Whittle, P. (1957). Curve and periodogram smoothing (with discussion). J. Roy. Statist. Soc. Ser. B 19 38–63. JSTOR: · Zbl 0089.35701
[33] Wong, W. H. and Shen, X. (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Statist. 23 339–362. · Zbl 0829.62002 · doi:10.1214/aos/1176324524
[34] Zhao, L. H. (2000). Bayesian aspects of some nonparametric problems. Ann. Statist. 28 532–552. · Zbl 1010.62025 · doi:10.1214/aos/1016218229
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.