×

Rates of contraction of posterior distributions based on Gaussian process priors. (English) Zbl 1141.60018

Summary: We derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on Gaussian processes. The rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel Hilbert space of the Gaussian process and the small ball probabilities of the Gaussian process. We determine these quantities for a range of examples of Gaussian priors and in several statistical settings. For instance, we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a (fractionally integrated) Brownian motion. We also consider regression with Gaussian errors and smooth classification under a logistic or probit link function combined with various priors.

MSC:

60G15 Gaussian processes
62G05 Nonparametric estimation
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Albert, J. H. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. J. Amer. Statist. Assoc. 88 422, 669-679. JSTOR: · Zbl 0774.62031 · doi:10.2307/2290350
[2] Ayache, A. and Taqqu, M. S. (2003). Rate optimality of wavelet series approximations of fractional Brownian motion. J. Fourier Anal. Appl. 9 451-471. · Zbl 1050.60043 · doi:10.1007/s00041-003-0022-0
[3] Borell, C. (1975). The Brunn-Minkowski inequality in Gauss space. Invent. Math. 30 207-216. · doi:10.1007/BF01425510
[4] Choi, T. and Schervish, M. (2004). Posterior consistency in nonparametric regression problem under Gaussian process prior. · Zbl 1138.62020 · doi:10.1016/j.jmva.2007.01.004
[5] Choudhuri, N., Ghosal, S. and Roy, A. (2007). Nonparametric binary regression using a Gaussian process prior. Statist. Methodol. 4 227-243. · Zbl 1248.62053 · doi:10.1016/j.stamet.2006.07.003
[6] Dzhaparidze, K. and van Zanten, H. (2004). A series expansion of fractional Brownian motion. Probab. Theory Related Fields 130 39-55. · Zbl 1059.60048 · doi:10.1007/s00440-003-0310-2
[7] Dzhaparidze, K. and van Zanten, H. (2005). Krein’s spectral theory and the Paley-Wiener expansion for fractional Brownian motion. Ann. Probab. 33 620-644. · Zbl 1083.60028 · doi:10.1214/009117904000000955
[8] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500-531. · Zbl 1105.62315 · doi:10.1214/aos/1016218228
[9] Ghosal, S. and Roy, A. (2006). Posterior consistency in nonparametric regression problem under Gaussian process prior. Ann. Statist. 34 2413-2429. · Zbl 1106.62039 · doi:10.1214/009053606000000795
[10] Ghosal, S. and van der Vaart, A. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities. Ann. Statist. 35 697-723. · Zbl 1117.62046 · doi:10.1214/009053606000001271
[11] Ghosal, S. and van der Vaart, A. (2007). Convergence rates of posterior distributions for noniid observations. Ann. Statist. 35 192-223. · Zbl 1114.62060 · doi:10.1214/009053606000001172
[12] Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics . Springer, New York. · Zbl 1029.62004
[13] Hult, H. (2003). Approximating some Volterra type stochastic integrals with applications to parameter estimation. Stochastic Process. Appl. 105 1-32. · Zbl 1075.60532 · doi:10.1016/S0304-4149(02)00250-8
[14] Iglói, E. (2005). A rate-optimal trigonometric series expansion of the fractional Brownian motion. Electron. J. Probab. 10 1381-1397 (electronic). · Zbl 1109.60032
[15] Kimeldorf, G. S. and Wahba, G. (1970). A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann. Math. Statist. 41 495-502. · Zbl 0193.45201 · doi:10.1214/aoms/1177697089
[16] Kuelbs, J., Li, W. V. and Linde, W. (1994). The Gaussian measure of shifted balls. Probab. Theory Related Fields 98 143-162. · Zbl 0792.60004 · doi:10.1007/BF01192511
[17] Kühn, T. and Linde, W. (2002). Optimal series representation of fractional Brownian sheets. Bernoulli 8 669-696. · Zbl 1012.60074
[18] Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces . Springer, Berlin. · Zbl 0748.60004
[19] Lenk, P. J. (1988). The logistic normal distribution for Bayesian, nonparametric, predictive densities. J. Amer. Statist. Assoc. 83 509-516. JSTOR: · Zbl 0648.62034 · doi:10.2307/2288870
[20] Lenk, P. J. (1991). Towards a practicable Bayesian nonparametric density estimator. Biometrika 78 531-543. JSTOR: · Zbl 0737.62035 · doi:10.1093/biomet/78.3.531
[21] Lenk, P. J. (1999). Bayesian inference for semiparametric regression using a Fourier representation. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 863-879. JSTOR: · Zbl 0940.62021 · doi:10.1111/1467-9868.00207
[22] Leonard, T. (1978). Density estimation, stochastic processes and prior information (with discussion). J. Roy. Statist. Soc. Ser. B 40 113-146. JSTOR: · Zbl 0398.62033
[23] Li, W. V. and Linde, W. (1998). Existence of small ball constants for fractional Brownian motions. C. R. Acad. Sci. Paris Sér. I Math. 326 1329-1334. · Zbl 0922.60039 · doi:10.1016/S0764-4442(98)80189-4
[24] Li, W. V. and Shao, Q.-M. (2001). Gaussian processes: Inequalities, small ball probabilities and applications. In Stochastic Processes : Theory and Methods. Handbook of Statist. 19 533-597. North-Holland, Amsterdam. · Zbl 0987.60053
[25] Mandelbrot, B. B. and Van Ness, J. W. (1968). Fractional Brownian motions, fractional noises and applications. SIAM Rev. 10 422-437. JSTOR: · Zbl 0179.47801 · doi:10.1137/1010093
[26] Neal, R. (1996). Bayesian Learning for Neural Networks. Lecture Notes in Statist. 118 . Springer, New York. · Zbl 0888.62021
[27] Rasmussen, C. E. and Williams, C. K. (2006). Gaussian Processes for Machine Learning . MIT Press, Cambridge, MA. · Zbl 1177.68165
[28] Samko, S. G., Kilbas, A. A. and Marichev, O. I. (1993). Fractional Integrals and Derivatives . Gordon and Breach, Yverdon. · Zbl 0818.26003
[29] Tokdar, S. and Ghosh, J. (2005). Posterior consistency of Gaussian process priors in density estimation. J. Statist. Plann. Inference 137 34-42. · Zbl 1098.62041 · doi:10.1016/j.jspi.2005.09.005
[30] van der Vaart, A. and Van Zanten, J. (2007). Reproducing kernel Hilbert spaces of Gaussian priors. In Festschrift for J. K. Ghosh. IMS Lecture Note Series , 2008 . · Zbl 1140.62066 · doi:10.1214/074921708000000156
[31] van der Vaart, A. W. (1988). Statistical Estimation in Large Parameter Spaces . Math. Centrum, Amsterdam. · Zbl 0629.62035
[32] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes . Springer, New York. · Zbl 0862.60002
[33] Wahba, G. (1978). Improper priors, spline smoothing and the problem of guarding against model errors in regression. J. Roy. Statist. Soc. Ser. B 40 364-372. JSTOR: · Zbl 0407.62048
[34] Wood, S. and Kohn, R. (1998). A Bayesian approach to robust nonparametric binary regression. J. Amer. Statist. Assoc. 93 203-213. · Zbl 0906.62037 · doi:10.2307/2669617
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.