×

Inferring causal impact using Bayesian structural time-series models. (English) Zbl 1454.62473

Summary: An important problem in econometrics and marketing is to infer the causal impact that a designed market intervention has exerted on an outcome metric over time. This paper proposes to infer causal impact on the basis of a diffusion-regression state-space model that predicts the counterfactual market response in a synthetic control that would have occurred had no intervention taken place. In contrast to classical difference-in-differences schemes, state-space models make it possible to (i) infer the temporal evolution of attributable impact, (ii) incorporate empirical priors on the parameters in a fully Bayesian treatment, and (iii) flexibly accommodate multiple sources of variation, including local trends, seasonality and the time-varying influence of contemporaneous covariates. Using a Markov chain Monte Carlo algorithm for posterior inference, we illustrate the statistical properties of our approach on simulated data. We then demonstrate its practical utility by estimating the causal effect of an online advertising campaign on search-related site visits. We discuss the strengths and limitations of state-space models in enabling causal attribution in those settings where a randomised experiment is unavailable. The CausalImpact R package provides an implementation of our approach.

MSC:

62P20 Applications of statistics to economics
62F15 Bayesian inference
62M20 Inference from stochastic processes and prediction

Software:

R; CausalImpact
PDF BibTeX XML Cite
Full Text: DOI arXiv Euclid

References:

[1] Abadie, A. (2005). Semiparametric difference-in-differences estimators. Rev. Econom. Stud. 72 1-19. · Zbl 1112.62132
[2] Abadie, A., Diamond, A. and Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. J. Amer. Statist. Assoc. 105 493-505. · Zbl 06445704
[3] Abadie, A. and Gardeazabal, J. (2003). The economic costs of conflict: A case study of the basque country. Amer. Econ. Rev. 93 113-132.
[4] Angrist, J. D. and Krueger, A. B. (1999). Empirical strategies in labor economics. Handbook of Labor Economics 3 1277-1366.
[5] Angrist, J. D. and Pischke, J.-S. (2008). Mostly Harmless Econometrics : An Empiricist’s Companion . Princeton Univ. Press, Princeton, NJ. · Zbl 1159.62090
[6] Antonakis, J., Bendahan, S., Jacquart, P. and Lalive, R. (2010). On making causal claims: A review and recommendations. Leadersh. Q. 21 1086-1120.
[7] Ashenfelter, O. and Card, D. (1985). Using the longitudinal structure of earnings to estimate the effect of training programs. Rev. Econ. Stat. 67 648-660.
[8] Ataman, M. B., Mela, C. F. and Van Heerde, H. J. (2008). Building brands. Mark. Sci. 27 1036-1054.
[9] Athey, S. and Imbens, G. W. (2002). Identification and inference in nonlinear difference-in-differences models. Working Paper 280, National Bureau of Economic Research, Cambridge, MA. · Zbl 1145.62316
[10] Banerjee, S., Kauffman, R. J. and Wang, B. (2007). Modeling Internet firm survival using Bayesian dynamic models with time-varying coefficients. Electron. Commer. Res. Appl. 6 332-342.
[11] Belloni, A., Chernozhukov, V., Fernandez-Val, I. and Hansen, C. (2013). Program evaluation with high-dimensional data. CeMMAP Working Paper CWP77/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies, London.
[12] Berndt, E. R. (1991). The Practice of Econometrics : Classic and Contemporary . Addison-Wesley, Reading, MA.
[13] Bertrand, M., Duflo, E. and Mullainathan, S. (2002). How much should we trust differences-in-differences estimates? Working Paper 8841, National Bureau of Economic Research, Cambridge, MA. · Zbl 1053.62132
[14] Brady, H. E. (2002). Models of causal inference: Going beyond the Neyman-Rubin-Holland theory. In Annual Meetings of the Political Methodology Group , Boston, MA.
[15] Brodersen, K. H., Daunizeau, J., Mathys, C., Chumbley, J. R., Buhmann, J. M. and Stephan, K. E. (2013). Variational Bayesian mixed-effects inference for classification studies. Neuroimage 76 345-361. · Zbl 1436.62243
[16] Camillo, F. and d’Attoma, I. (2010). A new data mining approach to estimate causal effects of policy interventions. Expert Syst. Appl. 37 171-181.
[17] Campbell, D. T., Stanley, J. C. and Gage, N. L. (1963). Experimental and Quasi-Experimental Designs for Research . Houghton Mifflin, Boston.
[18] Card, D. and Krueger, A. B. (1993). Minimum wages and employment: A case study of the fast food industry in New Jersey and Pennsylvania. Technical report, National Bureau of Economic Research, Cambridge, MA.
[19] Carter, C. K. and Kohn, R. (1994). On Gibbs sampling for state space models. Biometrika 81 541-553. · Zbl 0809.62087
[20] Chan, D., Ge, R., Gershony, O., Hesterberg, T. and Lambert, D. (2010). Evaluating online ad campaigns in a pipeline: Causal models at scale. In Proceedings of the 16 th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 7-16. ACM, New York.
[21] Chipman, H., George, E. I. and McCulloch, R. E. (2001). The practical implementation of Bayesian model selection. In Model Selection. Institute of Mathematical Statistics Lecture Notes-Monograph Series 38 65-134. IMS, Beachwood, OH.
[22] Claveau, F. (2012). The Russo-Williamson Theses in the social sciences: Causal inference drawing on two types of evidence. Stud. Hist. Philos. Biol. Biomed. Sci. 43 806-813.
[23] Cox, D. and Wermuth, N. (2001). Causal inference and statistical fallacies. In International Encyclopedia of the Social & Behavioral Sciences (Neil J. Smelser and P. B. Baltes, eds.) 1554-1561. Pergamon, Oxford.
[24] Danaher, P. J. and Rust, R. T. (1996). Determining the optimal return on investment for an advertising campaign. European J. Oper. Res. 95 511-521. · Zbl 0943.90591
[25] de Jong, P. and Shephard, N. (1995). The simulation smoother for time series models. Biometrika 82 339-350. · Zbl 0823.62072
[26] Donald, S. G. and Lang, K. (2007). Inference with difference-in-differences and other panel data. Rev. Econ. Stat. 89 221-233.
[27] Durbin, J. and Koopman, S. J. (2002). A simple and efficient simulation smoother for state space time series analysis. Biometrika 89 603-615. · Zbl 1036.62071
[28] Frühwirth-Schnatter, S. (1994). Data augmentation and dynamic linear models. J. Time Series Anal. 15 183-202. · Zbl 0815.62065
[29] George, E. I. and McCulloch, R. E. (1993). Variable selection via Gibbs sampling. J. Amer. Statist. Assoc. 88 881-889.
[30] George, E. I. and McCulloch, R. E. (1997). Approaches for Bayesian variable selection. Statist. Sinica 7 339-374. · Zbl 0884.62031
[31] Ghosh, J. and Clyde, M. A. (2011). Rao-Blackwellization for Bayesian variable selection and model averaging in linear and binary regression: A novel data augmentation approach. J. Amer. Statist. Assoc. 106 1041-1052. · Zbl 1229.62029
[32] Hansen, C. B. (2007a). Asymptotic properties of a robust variance matrix estimator for panel data when \(T\) is large. J. Econometrics 141 597-620. · Zbl 1418.62461
[33] Hansen, C. B. (2007b). Generalized least squares inference in panel and multilevel models with serial correlation and fixed effects. J. Econometrics 140 670-694. · Zbl 1247.91137
[34] Heckman, J. J. and Vytlacil, E. J. (2007). Econometric evaluation of social programs, Part I: Causal models, structural models and econometric policy evaluation. In Handbook of Econometrics 6, Part B (J. J. Heckman and E. E. Leamer, eds.) 4779-4874. Elsevier, Amsterdam.
[35] Hitchcock, C. (2004). Do all and only causes raise the probabilities of effects? In Causation and Counterfactuals . MIT Press, Cambridge.
[36] Hoover, K. D. (2012). Economic theory and causal inference. In Philosophy of Economics 13 (U. Mäki, ed.) 89-113. Elsevier, Amsterdam.
[37] Kleinberg, S. and Hripcsak, G. (2011). A review of causal inference for biomedical informatics. J. Biomed. Inform. 44 1102-1112.
[38] Leeflang, P. S., Bijmolt, T. H., van Doorn, J., Hanssens, D. M., van Heerde, H. J., Verhoef, P. C. and Wieringa, J. E. (2009). Creating lift versus building the base: Current trends in marketing dynamics. Int. J. Res. Mark. 26 13-20.
[39] Lester, R. A. (1946). Shortcomings of marginal analysis for wage-employment problems. Amer. Econ. Rev. 36 63-82.
[40] Lewis, R. A., Rao, J. M. and Reiley, D. H. (2011). Here, there, and everywhere: Correlated online behaviors can lead to overestimates of the effects of advertising. In Proceedings of the 20 th International Conference on World Wide Web. WWW’ 11 157-166. ACM, New York.
[41] Lewis, R. A. and Reiley, D. H. (2011). Does retail advertising work? Technical report.
[42] Liang, F., Paulo, R., Molina, G., Clyde, M. A. and Berger, J. O. (2008). Mixtures of \(g\) priors for Bayesian variable selection. J. Amer. Statist. Assoc. 103 410-423. · Zbl 1335.62026
[43] Mathys, C., Daunizeau, J., Friston, K. J. and Stephan, K. E. (2011). A Bayesian foundation for individual learning under uncertainty. Front. Human Neurosci. 5 39.
[44] Meyer, B. D. (1995). Natural and quasi-experiments in economics. J. Bus. Econom. Statist. 13 151.
[45] Morgan, S. L. and Winship, C. (2007). Counterfactuals and Causal Inference : Methods and Principles for Social Research . Cambridge Univ. Press, Cambridge.
[46] Nakajima, J. and West, M. (2013). Bayesian analysis of latent threshold dynamic models. J. Bus. Econom. Statist. 31 151-164.
[47] Polson, N. G. and Scott, S. L. (2011). Data augmentation for support vector machines. Bayesian Anal. 6 1-23. · Zbl 1330.62258
[48] Robinson, G., McNulty, J. E. and Krasno, J. S. (2009). Observing the counterfactual? The search for political experiments in nature. Polit. Anal. 17 341-357.
[49] Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66 688.
[50] Rubin, D. B. (2008). Statistical inference for causal effects, with emphasis on applications in epidemiology and medical statistics. In Epidemiology and Medical Statistics (J. P. Miller, C. R. Rao and D. C. Rao, eds.). Handbook of Statist. 27 28-63. Elsevier, Amsterdam.
[51] Rubin, D. B. and Waterman, R. P. (2006). Estimating the causal effects of marketing interventions using propensity score methodology. Statist. Sci. 21 206-222. · Zbl 1426.62325
[52] Scott, J. G. and Berger, J. O. (2010). Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. Ann. Statist. 38 2587-2619. · Zbl 1200.62020
[53] Scott, S. L. and Varian, H. R. (2014). Predicting the present with Bayesian structural time series. International Journal of Mathematical Modeling and Optimization 5 4-23. · Zbl 1302.62289
[54] Seggie, S. H., Cavusgil, E. and Phelan, S. E. (2007). Measurement of return on marketing investment: A conceptual framework and the future of marketing metrics. Ind. Mark. Manage. 36 834-841.
[55] Shadish, W. R., Cook, T. D. and Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference . Wadsworth Cengage Learning, Seattle, WA.
[56] Solon, G. (1984). Estimating Autocorrelations in Fixed-Effects Models . National Bureau of Economic Research, Cambridge, MA.
[57] Stewart, D. W. (2009). Marketing accountability: Linking marketing actions to financial results. J. Bus. Res. 62 636-643.
[58] Takada, H. and Bass, F. M. (1998). Multiple time series analysis of competitive marketing behavior. J. Bus. Res. 43 97-107.
[59] Vaver, J. and Koehler, J. (2011). Measuring ad effectiveness using geo experiments. Technical report, Google Inc.
[60] Vaver, J. and Koehler, J. (2012). Periodic measurement of advertising effectiveness using multiple-test-period geo experiments. Technical report, Google Inc.
[61] West, M. and Harrison, J. (1997). Bayesian Forecasting and Dynamic Models , 2nd ed. Springer, New York. · Zbl 0871.62026
[62] Winship, C. and Morgan, S. L. (1999). The estimation of causal effects from observational data. Annu. Rev. Sociol. 659-706.
[63] Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with \(g\)-prior distributions. In Bayesian Inference and Decision Techniques : Essays in Honor of Bruno de Finetti (P. K. Goel and A. Zellner, eds.). Stud. Bayesian Econometrics Statist. 6 233-243. North-Holland, Amsterdam. · Zbl 0655.62071
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.