Exact adaptive confidence intervals for linear regression coefficients. (English) Zbl 1461.62124

Hoff, Peter; Yu, Chaoyu

Electron. J. Stat. 13, No. 1, 94-119 (2019).

In this paper, the authors consider an alternative to the uniformly most accurate unbiased confidence interval procedure for the coefficients in the normal linear regression model \(\mathbf{y} \sim N_n(X\mathbf{\beta}, \sigma^2)\). They present an adaptive confidence interval procedure for adaptively estimating a normal prior distribution for \(\mathbf{\beta}\) from the data \(\mathbf{y}\) to reduce the potential risk of a poorly specified prior, and then using this estimated prior distribution to construct an approximately Bayes-optimal confidence interval procedure for each regression coefficient \(\beta_j,\) \(j = 1, \ldots, p\). The resulting adaptive confidence intervals maintain exact non-asymptotic \(1-\alpha\) coverage if the design matrix is full rank and the errors are normally distributed. Note that this procedure has a frequentist coverage rate that is constant as a function of the model parameters, yet provides smaller intervals than the usual interval procedure, on average across regression coefficients. No assumptions on the unknown parameters are necessary to maintain exact coverage. Additionally, in a “\(p\) growing with \(n\)” asymptotic scenario, this adaptive “frequentist, assisted by Bayes” procedure is asymptotically Bayes-optimal among \(1-\alpha\) frequentist confidence interval procedures.

Reviewer: Yuehua Wu (Toronto)

MSC:

62J05	Linear regression; mixed models
62J07	Ridge regression; shrinkage estimators (Lasso)
62G15	Nonparametric tolerance and confidence regions

Keywords:

adaptive confidence interval procedure; Bayes-optimal; empirical Bayes; frequentist coverage; ridge regression; shrinkage; sparsity; spending function; uniformly most accurate unbiased confidence interval procedure

References:

[1]	Andrews, D. W. K. (1992). Generic uniform convergence., Econometric Theory8(2), 241-257.
[2]	Bühlmann, P. (2013). Statistical significance in high-dimensional linear models., Bernoulli19(4), 1212-1242. · Zbl 1273.62173 · doi:10.3150/12-BEJSP11
[3]	Carlin, B. P. and A. E. Gelfand (1990). Approaches for empirical Bayes confidence intervals., J. Amer. Statist. Assoc.85(409), 105-114. · Zbl 0731.62080 · doi:10.1080/01621459.1990.10475312
[4]	Conlon, E. M., X. S. Liu, J. D. Lieb, and J. S. Liu (2003). Integrating regulatory motif discovery and genome-wide expression analysis., Proceedings of the National Academy of Sciences100(6), 3339-3344.
[5]	Efron, B., T. Hastie, I. Johnstone, and R. Tibshirani (2004). Least angle regression., Ann. Statist.32(2), 407-499. With discussion, and a rejoinder by the authors. · Zbl 1091.62054 · doi:10.1214/009053604000000067
[6]	Farchione, D. and P. Kabaila (2008). Confidence intervals for the normal mean utilizing prior information., Statist. Probab. Lett.78(9), 1094-1100. · Zbl 1137.62322 · doi:10.1016/j.spl.2007.11.003
[7]	Kabaila, P. and K. Giri (2009). Confidence intervals in regression utilizing prior information., J. Statist. Plann. Inference139(10), 3419-3429. · Zbl 1167.62027 · doi:10.1016/j.jspi.2009.03.018
[8]	Kabaila, P. and D. Tissera (2014). Confidence intervals in regression that utilize uncertain prior information about a vector parameter., Australian & New Zealand Journal of Statistics56(4), 371-383. · Zbl 1334.62112 · doi:10.1111/anzs.12090
[9]	LeCam, L. (1953). On some asymptotic properties of maximum likelihood estimates and related Bayes’ estimates., Univ. California Publ. Statist.1, 277-329. · Zbl 0052.15404
[10]	Lee, J. D., D. L. Sun, Y. Sun, and J. E. Taylor (2016). Exact post-selection inference, with application to the lasso., Ann. Statist.44(3), 907-927. · Zbl 1341.62061 · doi:10.1214/15-AOS1371
[11]	Leeb, H. and B. M. Pötscher (2008). Sparse estimators and the oracle property, or the return of Hodges’ estimator., J. Econometrics142(1), 201-211. · Zbl 1418.62272 · doi:10.1016/j.jeconom.2007.05.017
[12]	Meinshausen, N., L. Meier, and P. Bühlmann (2009). \(p\)-values for high-dimensional regression., J. Amer. Statist. Assoc.104(488), 1671-1681. · Zbl 1205.62089 · doi:10.1198/jasa.2009.tm08647
[13]	Mitchell, T. J. and J. J. Beauchamp (1988). Bayesian variable selection in linear regression., J. Amer. Statist. Assoc.83(404), 1023-1036. With comments by James Berger and C. L. Mallows and with a reply by the authors. · Zbl 0673.62051 · doi:10.1080/01621459.1988.10478694
[14]	Newey, W. K. and D. McFadden (1994). Large sample estimation and hypothesis testing. In, Handbook of econometrics, Vol. IV, Volume 2 of Handbooks in Econom., pp. 2111-2245. North-Holland, Amsterdam.
[15]	O’Gorman, T. W. (2001). Using adaptive weighted least squares to reduce the lengths of confidence intervals., Canad. J. Statist.29(3), 459-471. · Zbl 1004.62024
[16]	Pratt, J. W. (1963). Shorter confidence intervals for the mean of a normal distribution with known variance., The Annals of Mathematical Statistics34(2), 574-586. · Zbl 0114.35502 · doi:10.1214/aoms/1177704170
[17]	Puza, B. and T. O’Neill (2006). Interval estimation via tail functions., Canadian Journal of Statistics34(2), 299-310. · Zbl 1149.62016 · doi:10.1002/cjs.5550340207
[18]	Ranga Rao, R. (1962). Relations between weak and uniform convergence of measures with applications., Ann. Math. Statist.33, 659-680. · Zbl 0117.28602 · doi:10.1214/aoms/1177704588
[19]	Stein, C. M. (1962). Confidence sets for the mean of a multivariate normal distribution., J. Roy. Statist. Soc. Ser. B24, 265-296. · Zbl 0126.34602 · doi:10.1111/j.2517-6161.1962.tb00458.x
[20]	van de Geer, S., P. Bühlmann, Y. Ritov, and R. Dezeure (2014). On asymptotically optimal confidence regions and tests for high-dimensional models., Ann. Statist.42(3), 1166-1202. · Zbl 1305.62259 · doi:10.1214/14-AOS1221
[21]	Yoshimori, M. and P. Lahiri (2014). A second-order efficient empirical Bayes confidence interval., Ann. Statist.42(4), 1-29. · Zbl 1297.62019 · doi:10.1214/14-AOS1219
[22]	Yu, C. and P. D. Hoff (2018). Adaptive multigroup confidence intervals with constant coverage., Biometrika105(2), 319-335. · Zbl 07072415
[23]	Zhang, C.-H. and S. S. Zhang (2014). Confidence intervals for low dimensional parameters in high dimensional linear models., J. R. Stat. Soc. Ser. B. Stat. Methodol.76(1), 217-242. · Zbl 1411.62196