Rockova, Veronika; McAlinn, Kenichiro Dynamic variable selection with spike-and-slab process priors. (English) Zbl 1480.62132 Bayesian Anal. 16, No. 1, 233-269 (2021). Summary: We address the problem of dynamic variable selection in time series regression with unknown residual variances, where the set of active predictors is allowed to evolve over time. To capture time-varying variable selection uncertainty, we introduce new dynamic shrinkage priors for the time series of regression coefficients. These priors are characterized by two main ingredients: smooth parameter evolutions and intermittent zeroes for modeling predictive breaks. More formally, our proposed Dynamic Spike-and-Slab (DSS) priors are constructed as mixtures of two processes: a spike process for the irrelevant coefficients and a slab autoregressive process for the active coefficients. The mixing weights are themselves time-varying and depend on lagged values of the series. Our DSS priors are probabilistically coherent in the sense that their stationary distribution is fully known and characterized by spike-and-slab marginals. For posterior sampling over dynamic regression coefficients, model selection indicators as well as unknown dynamic residual variances, we propose a Dynamic SSVS algorithm based on forward-filtering and backward-sampling. To scale our method to large data sets, we develop a Dynamic EMVS algorithm for MAP smoothing. We demonstrate, through simulation and a topical macroeconomic dataset, that DSS priors are very effective at separating active and noisy coefficients. Our fast implementation significantly extends the reach of spike-and-slab methods to big time series data. Cited in 5 Documents MSC: 62H30 Classification and discrimination; cluster analysis (statistical aspects) 62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH) 62F07 Statistical ranking and selection procedures Keywords:autoregressive mixture processes; dynamic sparsity; MAP smoothing; spike and slab; stationarity Software:EMVS; bvarsv PDFBibTeX XMLCite \textit{V. Rockova} and \textit{K. McAlinn}, Bayesian Anal. 16, No. 1, 233--269 (2021; Zbl 1480.62132) Full Text: DOI arXiv Euclid References: [1] Andel, J. (1983). “Marginal distributions of autoregressive processes.” In Transactions of the Ninth Prague Conference, 127-135. Springer. · Zbl 0537.60027 [2] Antoniadis, A. and Fan, J. (2001). “Regularization of wavelet approximations.” Journal of the American Statistical Association, 96(455): 939-967. · Zbl 1072.62561 [3] Bai, J. and Ng, S. (2008). “Forecasting economic time series using targeted predictors.” Journal of Econometrics, 146(2): 304-317. · Zbl 1429.62419 [4] Banbura, M., Giannone, D., and Reichlin, L. (2010). “Large Bayesian vector auto regressions.” Journal of Applied Econometrics, 25(1): 71-92. [5] Barbieri, M. M. and Berger, J. O. (2004). “Optimal predictive model selection.” The Annals of Statistics, 32: 870-897. · Zbl 1092.62033 [6] Belmonte, M. A. G., Koop, G., and Korobilis, D. (2014). “Hierarchical shrinkage in time-varying parameter models.” Journal of Forecasting, 33(1): 80-94. · Zbl 1397.62090 [7] Bhattacharya, A., Pati, D., Pillai, N., and Dunson, D. (2015). “Dirichlet-Laplace priors for optimal shrinkage.” Journal of the American Statistical Association, 110: 1479-1490. · Zbl 1373.62368 [8] Bitto, A. and Frühwirth-Schnatter, S. (2019). “Achieving shrinkage in a time-varying parameter model framework.” Journal of Econometrics, 210: 75-97. · Zbl 1452.62216 [9] Brodie, J., Daubechies, I., De Mol, C., Giannone, D., and Loris, I. (2009). “Sparse and stable Markowitz portfolios.” Proceedings of the National Academy of Sciences, 106(30): 12267-12272. · Zbl 1203.91271 [10] Carlin, B. P. and Chib, S. (1995). “Bayesian model choice via Markov chain Monte Carlo methods.” Journal of the Royal Statistical Society. Series B (Methodological), 473-484. · Zbl 0827.62027 [11] Casarin, R., Dalla Valle, L., Leisen, F., et al. (2012a). “Bayesian model selection for beta autoregressive processes.” Bayesian Analysis, 7(2): 385-410. · Zbl 1330.62113 [12] Casarin, R., Valle, L., and Leisen, F. (2012b). “Bayesian model selection for beta autoregressive processes.” Bayesian Analysis, 7: 385-410. · Zbl 1330.62113 [13] Chan, J. C. C., Koop, G., Leon-Gonzalez, R., and Strachan, R. W. (2012). “Time varying dimension models.” Journal of Business & Economic Statistics, 30(3): 358-367. [14] Clyde, M., Desimone, H., and Parmigiani, G. (1996). “Prediction via orthogonalized model mixing.” Journal of the American Statistical Association, 91(435): 1197-1208. · Zbl 0880.62026 [15] Cogley, T. and Sargent, T. J. (2005). “Drifts and volatilities: Monetary policies and outcomes in the post WWII U.S.” Review of Economic Dynamics, 8: 262-302. [16] De Mol, C., Giannone, D., and Reichlin, L. (2008). “Forecasting using a large number of predictors: Is Bayesian shrinkage a valid alternative to principal components?” Journal of Econometrics, 146(2): 318-328. · Zbl 1429.62659 [17] Fan, J. and Li, R. (2001). “Variable selection via nonconcave penalized likelihood and its oracle properties.” Journal of the American statistical Association, 96(456): 1348-1360. · Zbl 1073.62547 [18] Frühwirth-Schnatter, S. (1994). “Data augmentation and dynamic linear models.” Journal of Time Series Analysis, 15: 183-202. · Zbl 0815.62065 [19] Frühwirth-Schnatter, S. and Wagner, H. (2010). “Stochastic model specification search for Gaussian and partial non-Gaussian state space models.” Journal of Econometrics, 154(1): 85-100. · Zbl 1431.62373 [20] Gefang, D. (2014). “Bayesian doubly adaptive elastic-net Lasso for VAR shrinkage.” International Journal of Forecasting, 30(1): 1-11. [21] George, E. I. (1986a). “Combining minimax shrinkage estimators.” Journal of the American Statistical Association, 81(394): 437-445. · Zbl 0594.62061 [22] George, E. I. (1986b). “Minimax multiple shrinkage estimation.” The Annals of Statistics, 14(1): 188-205. · Zbl 0602.62041 [23] George, E. I. and McCulloch, R. E. (1993). “Variable selection via Gibbs sampling.” Journal of the American Statistical Association, 88(423): 881-889. [24] George, E. I. and McCulloch, R. E. (1997). “Approaches for Bayesian variable selection.” Statistica Sinica, 339-373. · Zbl 0884.62031 [25] George, E. I., Sun, D., and Ni, S. (2008). “Bayesian stochastic search for VAR model restrictions.” Journal of Econometrics, 142(1): 553-580. · Zbl 1418.62322 [26] Giannone, D., Lenza, M., Momferatou, D., and Onorante, L. (2014). “Short-term inflation projections: a Bayesian vector autoregressive approach.” International Journal of Forecasting, 30(3): 635-644. [27] Godsill, S., Doucet, A., and West, M. (2001). “Maximum a posteriori sequence estimation using Monte Carlo particle filters.” Annals of the Institute of Statistical Mathematics, 53(1): 82-96. · Zbl 0995.62096 [28] Green, P. J. (1990). “On use of the EM for penalized likelihood estimation.” Journal of the Royal Statistical Society. Series B (Methodological), 443-452. · Zbl 0706.62022 [29] Groen, J. J. J., Paap, R., and Ravazzolo, F. (2013). “Real-time inflation forecasting in a changing world.” Journal of Business & Economic Statistics, 31(1): 29-44. [30] Hastie, T. and Tibshirani, R. (1993). “Varying-coefficient models.” Journal of the Royal Statistical Society. Series B (Methodological), 55: 757-796. · Zbl 0796.62060 [31] Irie, K. and West, M. (2016). “Bayesian Emulation for Optimization in Multi-Step Portfolio Decisions.” Submitted manuscript. · Zbl 1409.62173 [32] Ishwaran, H. and Rao, J. S. (2005). “Spike and slab variable selection: frequentist and Bayesian strategies.” The Annals of Statistics, 33: 730-773. · Zbl 1068.62079 [33] Jagannathan, R. and Ma, T. (2003). “Risk reduction in large portfolios: Why imposing the wrong constraints helps.” The Journal of Finance, 58: 1651-1684. [34] Kalli, M. and Griffin, J. E. (2014). “Time-varying sparsity in dynamic regression models.” Journal of Econometrics, 178: 779-793. · Zbl 1293.62191 [35] Kalliovirta, L., Meitz, M., and Saikkonen, P. (2015). “A Gaussian mixture autoregressive model for univariate time series.” Journal of Time Series Analysis, 36(2): 247-266. · Zbl 1320.62201 [36] Kastner, G., Frühwirth-Schnatter, S., and Lopes, H. (2017). “Efficient Bayesian inference for multivariate factor stochastic volatility models.” Journal of Computational and Graphical Statistics, 26: 905-917. [37] Kim, S., Shephard, N., and Chib, S. (1998). “Stochastic volatility: Likelihood inference and comparison with ARCH Models.” Review of Economic Studies, 65: 361-393. · Zbl 0910.90067 [38] Koop, G. and Korobilis, D. (2012a). “Forecasting inflation using dynamic model averaging.” International Economic Review, 53(3): 867-886. · Zbl 1420.91381 [39] Koop, G. and Korobilis, D. (2012b). “Forecasting inflation using dynamic model averaging.” International Econometrics Reviews, 30: 867-886. · Zbl 1420.91381 [40] Korobilis, D. (2013). “VAR forecasting using Bayesian variable selection.” Journal of Applied Econometrics, 28(2): 204-230. [41] Kowal, D., Matteson, D., and Ruppert, D. (2019). “Dynamic shrinkage processes.” Journal of the Royal Statistical Association (to appear), 1-30. · Zbl 1428.62397 [42] Li, J. and Chen, W. (2014). “Forecasting macroeconomic time series: Lasso-based approaches and their forecast combinations with dynamic factor models.” International Journal of Forecasting, 30(4): 996-1015. [43] Lopes, H., McCulloch, R., and Tsay, R. (2016). “Parsimony inducing priors for large scale state-space models.” Submitted. [44] McKenzie, E. (1985). “An Autoregressive process for beta random variables.” Management Science, 31: 988-995. · Zbl 0607.62104 [45] Meng, X. (1993). “Maximum likelihood estimation via the ECM algorithm: A general framework.” Biometrika, 80: 267-278. · Zbl 0778.62022 [46] Mitchell, T. J. and Beauchamp, J. J. (1988). “Bayesian variable selection in linear regression.” Journal of the American Statistical Association, 83: 1023-1032. · Zbl 0673.62051 [47] Nakajima, J. and West, M. (2013a). “Bayesian analysis of latent threshold dynamic models.” Journal of Business & Economic Statistics, 31: 151-164. [48] Nakajima, J. and West, M. (2013b). “Bayesian analysis of latent threshold dynamic models.” Journal of Business & Economic Statistics, 31: 151-164. [49] Park, T. and Casella, G. (2008). “The Bayesian LASSO.” Journal of the American Statistical Association, 103: 681-686. · Zbl 1330.62292 [50] Pettenuzzo, D., Koop, G., Korobilis, D., et al. (2018). “Bayesian compressed vector autoregressions.” Journal of Econometrics, Forthcoming. · Zbl 1452.62934 [51] Phillips, P. (1991). “To Criticize the Critics: An Objective Bayesian Analysis of Stochastic Trends.” Journal of Applied Econometrics, 6: 333-364. [52] Prado, R. and West, M. (2010). Time Series: Modelling, Computation & Inference. Chapman & Hall/CRC Press. · Zbl 1245.62105 [53] Primiceri, G. E. (2005). “Time varying structural vector autoregressions and monetary policy.” Review of Economic Studies, 72: 821-852. · Zbl 1106.91047 [54] Puelz, D., Hahn, P. R., and Carvalho, C. (2016). “Sparse mean-variance portfolios: A penalized utility approach.” Submitted manuscript. [55] Rockova, V. (2018). “Bayesian Estimation of Sparse Signals with a continuous spike-and-slab prior.” The Annals of Statistics, 46: 401-437. · Zbl 1395.62230 [56] Rockova, V. and George, E. I. (2014). “EMVS: The EM approach to Bayesian variable selection.” Journal of the American Statistical Association, 109: 828-846. · Zbl 1367.62049 [57] Rockova, V. and George, E. I. (2018). “The spike-and-slab LASSO.” Journal of the American Statistical Association (to appear), 113: 431-444. · Zbl 1398.62186 [58] Rockova, V. and McAlinn, K. (2020). “Dynamic Variable Selection with Spike-and-Slab Process Priors - Supplementary Material.” Bayesian Analysis. [59] Schotman, P. and van Dijk, H. (1991). “A Bayesian analysis of the unit root in real exchange rates.” Journal of Econometrics, 49: 195-238. · Zbl 0724.62105 [60] Stock, J. H. and Watson, M. W. (1999). “Forecasting inflation.” Journal of Monetary Economics, 44(2): 293-335. [61] Stock, J. H. and Watson, M. W. (2007). “Why has US inflation become harder to forecast?” Journal of Money, Credit and Banking, 39: 3-33. [62] Stock, J. H. and Watson, M. W. (2012). “Generalized shrinkage methods for forecasting using many predictors.” Journal of Business & Economic Statistics, 30(4): 481-493. [63] Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., and Knight, K. (2005). “Sparsity and smoothness via the fused LASSO.” Journal of the Royal Statistical Society. Series B, 67(1): 91-108. · Zbl 1060.62049 [64] West, M. and Harrison, J. (1997). Bayesian Forecasting & Dynamic Models. Springer Verlag, 2nd edition. · Zbl 0871.62026 [65] Wong, C. S. and Li, W. K. (2000). “On a mixture autoregressive model.” Journal of the Royal Statistical Society: Series B, 62(1): 95-115. · Zbl 0941.62095 [66] Wong, C. S. and Li, W. K. (2001). “On a logistic mixture autoregressive model.” Biometrika, 88(1): 833-846. · Zbl 0985.62074 [67] Wood, S., Rosen, O., and Kohn, R. (2011). “Bayesian mixtures of autoregressive models.” Journal of Computational and Graphical Statistics, 20(2): 174-195. [68] Wright, J. H. (2009). “Forecasting US inflation by Bayesian model averaging.” Journal of Forecasting, 28(2): 131-144. [69] Zellner, A. (1971). An Introduction to Bayesian Inference in Econometrics. Wiley, New York. · Zbl 0246.62098 [70] Zhang, C.-H. (2010). “Nearly unbiased variable selection under minimax concave penalty.” The Annals of Statistics, 38: 894-942. · Zbl 1183.62120 [71] Zhang, C.-H. and Zhang, T. (2012). “A general theory of concave regularization for high-dimensional sparse estimation problems.” Statistical Science, 27: 576-593. · Zbl 1331.62353 [72] Zou, H. · Zbl 1069.62054 This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.