×

Empirical Bayes oracle uncertainty quantification for regression. (English) Zbl 1461.62120

Summary: We propose an empirical Bayes method for high-dimensional linear regression models. Following an oracle approach that quantifies the error locally for each possible value of the parameter, we show that an empirical Bayes posterior contracts at the optimal rate at all parameters and leads to uniform size-optimal credible balls with guaranteed coverage under an “excessive bias restriction” condition. This condition gives rise to a new slicing of the entire space that is suitable for ensuring uniformity in uncertainty quantification. The obtained results immediately lead to optimal contraction and coverage properties for many conceivable classes simultaneously. The results are also extended to high-dimensional additive nonparametric regression models.

MSC:

62J05 Linear regression; mixed models
62C12 Empirical decision procedures; empirical Bayes procedures
62G08 Nonparametric regression and quantile regression
62G15 Nonparametric tolerance and confidence regions
PDFBibTeX XMLCite
Full Text: DOI Euclid

References:

[1] Baraud, Y. (2004). Confidence balls in Gaussian regression. Ann. Statist. 32 528-551. · Zbl 1093.62051 · doi:10.1214/009053604000000085
[2] Belitser, E. (2017). On coverage and local radial rates of credible sets. Ann. Statist. 45 1124-1151. · Zbl 1371.62044 · doi:10.1214/16-AOS1477
[3] Belitser, E. and Ghosal, S. (2020). Supplement to “Empirical Bayes oracle uncertainty quantification for regression.” https://doi.org/10.1214/19-AOS1845SUPP
[4] Belitser, E. and Nurushev, N. (2020). Needles and straw in a haystack: Robust empirical Bayes confidence for possibly sparse sequences. Bernoulli 26 191-225. · Zbl 1441.62110 · doi:10.3150/19-BEJ1122
[5] Birgé, L. and Massart, P. (2001). Gaussian model selection. J. Eur. Math. Soc. (JEMS) 3 203-268. · Zbl 1037.62001
[6] Bühlmann, P. (2013). Statistical significance in high-dimensional linear models. Bernoulli 19 1212-1242. · Zbl 1273.62173
[7] Cai, T. T. and Guo, Z. (2017). Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity. Ann. Statist. 45 615-646. · Zbl 1371.62045 · doi:10.1214/16-AOS1461
[8] Cai, T. T. and Low, M. G. (2004). An adaptation theory for nonparametric confidence intervals. Ann. Statist. 32 1805-1840. · Zbl 1056.62060 · doi:10.1214/009053604000000049
[9] Castillo, I. and Nickl, R. (2013). Nonparametric Bernstein-von Mises theorems in Gaussian white noise. Ann. Statist. 41 1999-2028. · Zbl 1285.62052 · doi:10.1214/13-AOS1133
[10] Castillo, I. and Nickl, R. (2014). On the Bernstein-von Mises phenomenon for nonparametric Bayes procedures. Ann. Statist. 42 1941-1969. · Zbl 1305.62190 · doi:10.1214/14-AOS1246
[11] Castillo, I., Schmidt-Hieber, J. and van der Vaart, A. (2015). Bayesian linear regression with sparse priors. Ann. Statist. 43 1986-2018. · Zbl 1486.62197 · doi:10.1214/15-AOS1334
[12] Castillo, I. and Szabó, B. (2020). Spike and slab empirical Bayes sparse credible sets. Bernoulli 26 127-158. · Zbl 1441.62077 · doi:10.3150/19-BEJ1119
[13] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903-923. · Zbl 0778.62003 · doi:10.1214/aos/1176349157
[14] Curtis, S. M., Banerjee, S. and Ghosal, S. (2014). Fast Bayesian model assessment for nonparametric additive regression. Comput. Statist. Data Anal. 71 347-358. · Zbl 1471.62136 · doi:10.1016/j.csda.2013.05.012
[15] Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425-455. · Zbl 0815.62019 · doi:10.1093/biomet/81.3.425
[16] Freedman, D. (1999). On the Bernstein-von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119-1140. · Zbl 0957.62002
[17] Ghosal, S. and van der Vaart, A. (2017). Fundamentals of Nonparametric Bayesian Inference. Cambridge Series in Statistical and Probabilistic Mathematics 44. Cambridge Univ. Press, Cambridge. · Zbl 1376.62004
[18] Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Monographs on Statistics and Applied Probability 43. CRC Press, London.
[19] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011). Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 2626-2657. · Zbl 1232.62079 · doi:10.1214/11-AOS920
[20] Li, K.-C. (1989). Honest confidence regions for nonparametric regression. Ann. Statist. 17 1001-1008. · Zbl 0681.62047 · doi:10.1214/aos/1176347253
[21] Lin, Y. and Zhang, H. H. (2006). Component selection and smoothing in multivariate nonparametric regression. Ann. Statist. 34 2272-2297. · Zbl 1106.62041 · doi:10.1214/009053606000000722
[22] Martin, R., Mess, R. and Walker, S. G. (2017). Empirical Bayes posterior concentration in sparse high-dimensional linear models. Bernoulli 23 1822-1847. · Zbl 1450.62085 · doi:10.3150/15-BEJ797
[23] Meier, L., van de Geer, S. and Bühlmann, P. (2009). High-dimensional additive modeling. Ann. Statist. 37 3779-3821. · Zbl 1360.62186 · doi:10.1214/09-AOS692
[24] Nickl, R. and van de Geer, S. (2013). Confidence sets in sparse regression. Ann. Statist. 41 2852-2876. · Zbl 1288.62108 · doi:10.1214/13-AOS1170
[25] Picard, D. and Tribouley, K. (2000). Adaptive confidence interval for pointwise curve estimation. Ann. Statist. 28 298-335. · Zbl 1106.62331 · doi:10.1214/aos/1016120374
[26] Raskutti, G., Wainwright, M. J. and Yu, B. (2012). Minimax-optimal rates for sparse additive models over kernel classes via convex programming. J. Mach. Learn. Res. 13 389-427. · Zbl 1283.62071
[27] Ravikumar, P., Lafferty, J., Liu, H. and Wasserman, L. (2009). Sparse additive models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 71 1009-1030. · Zbl 1411.62107 · doi:10.1111/j.1467-9868.2009.00718.x
[28] Ray, K. (2017). Adaptive Bernstein-von Mises theorems in Gaussian white noise. Ann. Statist. 45 2511-2536. · Zbl 1384.62158 · doi:10.1214/16-AOS1533
[29] Sniekers, S. and van der Vaart, A. (2015). Credible sets in the fixed design model with Brownian motion prior. J. Statist. Plann. Inference 166 78-86. · Zbl 1394.62048 · doi:10.1016/j.jspi.2014.07.008
[30] Sniekers, S. and van der Vaart, A. (2015). Adaptive Bayesian credible sets in regression with a Gaussian process prior. Electron. J. Stat. 9 2475-2527. · Zbl 1327.62300 · doi:10.1214/15-EJS1078
[31] Szabó, B., van der Vaart, A. W. and van Zanten, J. H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets. Ann. Statist. 43 1391-1428. · Zbl 1317.62040 · doi:10.1214/14-AOS1270
[32] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267-288. · Zbl 0850.62538 · doi:10.1111/j.2517-6161.1996.tb02080.x
[33] van de Geer, S. A. and Bühlmann, P. (2009). On the conditions used to prove oracle results for the Lasso. Electron. J. Stat. 3 1360-1392. · Zbl 1327.62425 · doi:10.1214/09-EJS506
[34] Yang, Y. and Tokdar, S. T. (2015). Minimax-optimal nonparametric regression in high dimensions. Ann. Statist. 43 652-674. · Zbl 1312.62052 · doi:10.1214/14-AOS1289
[35] Yoo, W. · Zbl 1338.62121 · doi:10.1214/15-AOS1398
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.