×

Inference after model averaging in linear regression models. (English) Zbl 1420.62300

Summary: This article considers the problem of inference for nested least squares averaging estimators. We study the asymptotic behavior of the Mallows model averaging estimator (MMA; [B. E. Hansen, Econometrica 75, No. 4, 1175–1189 (2007; Zbl 1133.91051)]) and the jackknife model averaging estimator (JMA; [B. E. Hansen and J. S. Racine, J. Econom. 167, No. 1, 38–46 (2012; Zbl 1441.62721)]) under the standard asymptotics with fixed parameters setup. We find that both MMA and JMA estimators asymptotically assign zero weight to the under-fitted models, and MMA and JMA weights of just-fitted and over-fitted models are asymptotically random. Building on the asymptotic behavior of model weights, we derive the asymptotic distributions of MMA and JMA estimators and propose a simulation-based confidence interval for the least squares averaging estimator. Monte Carlo simulations show that the coverage probabilities of proposed confidence intervals achieve the nominal level.

MSC:

62J05 Linear regression; mixed models
62G15 Nonparametric tolerance and confidence regions
62E20 Asymptotic distribution theory in statistics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] AndrewsD.W.K. (1991) Asymptotic optimality of generalized C_L, cross-validation, and generalized cross-validation in regression with heteroskedastic errors. Journal of Econometrics47, 359-377.10.1016/0304-4076(91)90107-O · Zbl 0734.62069
[2] BreimanL. (1996) Bagging predictors. Machine Learning24, 123-140.10.1007/BF00058655 · Zbl 0858.68080
[3] BucklandS.T., BurnhamK.P., & AugustinN.H. (1997) Model selection: An integral part of inference. Biometrics53, 603-618.10.2307/2533961 · Zbl 0885.62118
[4] CamponovoL. (2015) On the validity of the pairs bootstrap for lasso estimators. Biometrika102(4), 981-987.10.1093/biomet/asv039 · Zbl 1372.62021
[5] ChatterjeeA. & LahiriS.N. (2011) Bootstrapping lasso estimators. Journal of the American Statistical Association106(494), 608-625.10.1198/jasa.2011.tm10159 · Zbl 1232.62088
[6] ChatterjeeA. & LahiriS.N. (2013) Rates of convergence of the adaptive lasso estimators to the oracle distribution and higher order refinements by the bootstrap. The Annals of Statistics41(3), 1232-1259.10.1214/13-AOS1106 · Zbl 1293.62153
[7] ClaeskensG. & HjortN.L. (2008) Model Selection and Model Averaging. Cambridge University Press.10.1017/CBO9780511790485 · Zbl 1166.62001
[8] DiTragliaF. (2016) Using invalid instruments on purpose: Focused moment selection and averaging for GMM. Journal of Econometrics195, 187-208.10.1016/j.jeconom.2016.07.006 · Zbl 1443.62441
[9] HansenB.E. (2007) Least squares model averaging. Econometrica75, 1175-1189.10.1111/j.1468-0262.2007.00785.x · Zbl 1133.91051
[10] HansenB.E. (2008) Least-squares forecast averaging. Journal of Econometrics146(2), 342-350.10.1016/j.jeconom.2008.08.022 · Zbl 1429.62421
[11] HansenB.E. (2014) Model averaging, asymptotic risk, and regressor groups. Quantitative Economics5(3), 495-530.10.3982/QE332 · Zbl 1398.62181
[12] HansenB.E. (2018) Econometrics. Unpublished manuscript, University of Wisconsin.
[13] HansenB.E. & RacineJ. (2012) Jacknife model averaging. Journal of Econometrics167, 38-46.10.1016/j.jeconom.2011.06.019 · Zbl 1441.62721
[14] HansenP., LundeA., & NasonJ. (2011) The model confidence set. Econometrica79, 453-497. · Zbl 1210.62030
[15] HjortN.L. & ClaeskensG. (2003a) Frequentist model average estimators. Journal of the American Statistical Association98, 879-899.10.1198/016214503000000828 · Zbl 1047.62003
[16] HjortN.L. & ClaeskensG. (2003b) Rejoinder to the focused information criterion and frequentist model average estimators. Journal of the American Statistical Association98(464), 938-945.10.1198/016214503000000882
[17] HoetingJ.A., MadiganD., RafteryA.E., & VolinskyC.T. (1999) Bayesian model averaging: A tutorial. Statistical Science14, 382-417. · Zbl 1059.62525
[18] InoueA. & KilianL. (2008) How useful is bagging in forecasting economic time series? A case study of US consumer price inflation. Journal of the American Statistical Association103, 511-522.10.1198/016214507000000473 · Zbl 1469.62403
[19] KabailaP. (1995) The effect of model selection on confidence regions and prediction regions. Econometric Theory11(3), 537-549.10.1017/S0266466600009403
[20] KabailaP. (1998) Valid confidence intervals in regression after variable selection. Econometric Theory14(4), 463-482.10.1017/S0266466698144031
[21] KimJ. & PollardD. (1990) Cube root asymptotics. The Annals of Statistics18, 191-219.10.1214/aos/1176347498 · Zbl 0703.62063
[22] LeebH. & PötscherB. (2003) The finite-sample distribution of post-model-selection estimators and uniform versus non-uniform approximations. Econometric Theory19(1), 100-142.10.1017/S0266466603191050 · Zbl 1032.62011
[23] LeebH. & PötscherB. (2005) Model selection and inference: Facts and fiction. Econometric Theory21(1), 21-59.10.1017/S0266466605050036 · Zbl 1085.62004
[24] LeebH. & PötscherB. (2006) Can one estimate the conditional distribution of post-model-selection estimators?The Annals of Statistics34(5), 2554-2591.10.1214/009053606000000821 · Zbl 1106.62029
[25] LeebH. & PötscherB. (2008) Can one estimate the unconditional distribution of post-model-selection estimators?Econometric Theory24(02), 338-376.10.1017/S0266466608080158 · Zbl 1284.62152
[26] LeebH. & PötscherB. (2017) Testing in the presence of nuisance parameters: Some comments on tests post-model-selection and random critical values. In AhmedS.E. (ed.), Big and Complex Data Analysis: Methodologies and Applications, pp. 69-82. Springer International Publishing.10.1007/978-3-319-41573-4_4 · Zbl 1380.62089
[27] LiK.-C. (1987) Asymptotic optimality for C_p, C_L, cross-validation and generalized cross-validation: Discrete index set. The Annals of Statistics15, 958-975.10.1214/aos/1176350486 · Zbl 0653.62037
[28] LiangH., ZouG., WanA.T.K., & ZhangX. (2011) Optimal weight choice for frequentist model average estimators. Journal of the American Statistical Association106, 1053-1066.10.1198/jasa.2011.tm09478 · Zbl 1229.62090
[29] LiuC.-A. (2015) Distribution theory of the least squares averaging estimator. Journal of Econometrics186, 142-159.10.1016/j.jeconom.2014.07.002 · Zbl 1331.62337
[30] LiuQ. & OkuiR. (2013) Heteroscedasticity-robust C_p model averaging. Econometrics Journal16, 462-473.10.1111/ectj.12009
[31] LuX. (2015) A covariate selection criterion for estimation of treatment effects. Journal of Business and Economic Statistics33, 506-522.10.1080/07350015.2014.982755
[32] LuX. & SuL. (2015) Jackknife model averaging for quantile regressions. Journal of Econometrics188(1), 40-58.10.1016/j.jeconom.2014.11.005 · Zbl 1337.62080
[33] MagnusJ., PowellO., & PrüferP. (2010) A comparison of two model averaging techniques with an application to growth empirics. Journal of Econometrics154(2), 139-153.10.1016/j.jeconom.2009.07.004 · Zbl 1431.62654
[34] Moral-BenitoE. (2015) Model averaging in economics: An overview. Journal of Economic Surveys29(1), 46-75.10.1111/joes.12044
[35] PötscherB. (1991) Effects of model selection on inference. Econometric Theory7(2), 163-185.10.1017/S0266466600004382
[36] PötscherB. (2006) The distribution of model averaging estimators and an impossibility result regarding its estimation. Lecture Notes-Monograph Series52, 113-129.10.1214/074921706000000987 · Zbl 1268.62066
[37] PötscherB. & LeebH. (2009) On the distribution of penalized maximum likelihood estimators: The lasso, scad, and thresholding. Journal of Multivariate Analysis100(9), 2065-2082.10.1016/j.jmva.2009.06.010 · Zbl 1170.62046
[38] RafteryA.E. & ZhengY. (2003) Discussion: Performance of Bayesian model averaging. Journal of the American Statistical Association98(464), 931-938.10.1198/016214503000000891
[39] TibshiraniR. (1996) Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B58, 267-288. · Zbl 0850.62538
[40] Van der VaartA. & WellnerJ. (1996) Weak Convergence and Empirical Processes. Springer Verlag.10.1007/978-1-4757-2545-2 · Zbl 0862.60002
[41] WanA.T.K., ZhangX., & ZouG. (2010) Least squares model averaging by Mallows criterion. Journal of Econometrics156, 277-283.10.1016/j.jeconom.2009.10.030 · Zbl 1431.62291
[42] YangY. (2000) Combining different procedures for adaptive regression. Journal of Multivariate Analysis74(1), 135-161.10.1006/jmva.1999.1884 · Zbl 0964.62032
[43] YangY. (2001) Adaptive regression by mixing. Journal of the American Statistical Association96, 574-588.10.1198/016214501753168262 · Zbl 1018.62033
[44] YuanZ. & YangY. (2005) Combining linear regression models: When and how?Journal of the American Statistical Association100, 1202-1214.10.1198/016214505000000088 · Zbl 1117.62454
[45] ZhangX. & LiangH. (2011) Focused information criterion and model averaging for generalized additive partial linear models. The Annals of Statistics39, 174-200.10.1214/10-AOS832 · Zbl 1209.62088
[46] ZhangX., WanA.T., & ZhouS.Z. (2012) Focused information criteria, model selection, and model averaging in a Tobit model with a nonzero threshold. Journal of Business and Economic Statistics30, 132-142.10.1198/jbes.2011.10075
[47] ZhangX., WanA.T.K., & ZouG. (2013) Model averaging by jackknife criterion in models with dependent data. Journal of Econometrics174, 82-94.10.1016/j.jeconom.2013.01.004 · Zbl 1283.62059
[48] ZhangX., ZouG., & LiangH. (2014) Model averaging and weight choice in linear mixed-effects models. Biometrika101, 205-218.10.1093/biomet/ast052 · Zbl 1285.62077
[49] ZouH. (2006) The adaptive lasso and its oracle properties. Journal of the American Statistical Association101, 1418-1429.10.1198/016214506000000735 · Zbl 1171.62326
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.