Keele, Luke; Morgan, Jason W. How strong is strong enough? Strengthening instruments through matching and weak instrument tests. (English) Zbl 1398.62381 Ann. Appl. Stat. 10, No. 2, 1086-1106 (2016). Summary: In a natural experiment, treatment assignments are made through a haphazard process that is thought to be as-if random. In one form of the natural experiment, encouragement to accept treatment rather than treatments themselves are assigned in this haphazard process. This encouragement to accept treatment is often referred to as an instrument. Instruments can be characterized by different levels of strength depending on the amount of encouragement. Weak instruments that provide little encouragement may produce biased inferences, particularly when assignment of the instrument is not strictly randomized. A specialized matching algorithm can be used to strengthen instruments by selecting a subset of matched pairs where encouragement is strongest. We demonstrate how weak instrument tests can guide the matching process to ensure that the instrument has been sufficiently strengthened. Specifically, we combine a matching algorithm for strengthening instruments and weak instrument tests in the context of a study of whether turnout influences party vote share in US elections. It is thought that when turnout is higher, Democratic candidates will receive a higher vote share. Using excess rainfall as an instrument, we hope to observe an instance where unusually wet weather produces lower turnout in an as-if random fashion. Consistent with statistical theory, we find that strengthening the instrument reduces sensitivity to bias from an unobserved confounder. Cited in 7 Documents MSC: 62P25 Applications of statistics to social sciences Keywords:causal inference; instrumental variables; matching; weak instruments Software:sensitivitymv; DOS × Cite Format Result Cite Review PDF Full Text: DOI Euclid References: [1] Angrist, J. D., Imbens, G. W. and Rubin, D. B. (1996). Identification of causal effects using instrumental variables. J. Amer. Statist. Assoc. 91 444-455. · Zbl 0897.62130 · doi:10.2307/2291629 [2] Angrist, J. D. and Pischke, J.-S. (2010). The credibility revolution in empirical economics: How better research design is taking the con out of econometrics. J. Econ. Perspect. 24 3-30. [3] Baiocchi, M., Small, D. S., Lorch, S. and Rosenbaum, P. R. (2010). Building a stronger instrument in an observational study of perinatal care for premature infants. J. Amer. Statist. Assoc. 105 1285-1296. · Zbl 1388.62311 · doi:10.1198/jasa.2010.ap09490 [4] Baiocchi, M., Small, D. S., Yang, L., Polsky, D. and Groeneveld, P. W. (2012). Near/far matching: A study design approach to instrumental variables. Health Serv. Outcomes Res. Methodol. 12 237-253. [5] Bound, J., Jaeger, D. A. and Baker, R. M. (1995). Problems with intrustmental variables estimation when the correlation between the instruments and the endogenous explanatory variable is weak. J. Amer. Statist. Assoc. 90 443-450. [6] Brumback, B. A., Hernán, M. A., Haneuse, S. J. and Robins, J. M. (2004). Sensitivity analyses for unmeasured confounding assuming a marginal structural model for repeated measures. Stat. Med. 23 749-767. [7] Chamberlain, G. and Imbens, G. (2004). Random effects estimators with many instrumental variables. Econometrica 72 295-306. · Zbl 1140.62353 · doi:10.1111/j.1468-0262.2004.00485.x [8] Chao, J. C. and Swanson, N. R. (2005). Consistent estimation with a large number of weak instruments. Econometrica 73 1673-1692. · Zbl 1151.62366 · doi:10.1111/j.1468-0262.2005.00632.x [9] Cornfield, J., Haenszel, W., Hammond, E., Lilienfeld, A., Shimkin, M. and Wynder, E. (1959). Smoking and lung cancer: Recent evidence and a discussion of some questions. J. Natl. Cancer Inst. 22 173-203. [10] Dunning, T. (2012). Natural Experiments in the Social Sciences : A Design-Based Approach . Cambridge Univ. Press, Cambridge, UK. [11] Gomez, B. T., Hansford, T. G. and Krause, G. A. (2007). The republicans should pray for rain: Weather turnout, and voting in U.S. presidential elections. J. Polit. 69 649-663. [12] Hansford, T. G. and Gomez, B. T. (2010). Estimating the electoral effects of voter turnout. Am. Polit. Sci. Rev. 104 268-288. [13] Hodges, J. L. Jr. and Lehmann, E. L. (1963). Estimates of location based on rank tests. Ann. Math. Stat. 34 598-611. · Zbl 0203.21105 · doi:10.1214/aoms/1177704172 [14] Holland, P. W. (1988). Causal inference, path analysis, and recursive structural equation models. Sociol. Method. 18 449-484. [15] Imbens, G. W. and Angrist, J. D. (1994). Identification and estimation of local average treatment effects. Econometrica 62 467-476. · Zbl 0800.90648 · doi:10.2307/2951620 [16] Imbens, G. W. and Rosenbaum, P. R. (2005). Robust, accurate confidence intervals with a weak instrument: Quarter of birth and education. J. Roy. Statist. Soc. Ser. A 168 109-126. · Zbl 1101.62120 · doi:10.1111/j.1467-985X.2004.00339.x [17] Keele, L. J. and Minozzi, W. (2012). How much is Minnesota like Wisconsin? Assumptions and counterfactuals in causal inference with observational data. Polit. Anal. 21 193-216. [18] Keele, L., Titiunik, R. and Zubizarreta, J. R. (2015). Enhancing a geographic regression discontinuity design through matching to estimate the effect of ballot initiatives on voter turnout. J. Roy. Statist. Soc. Ser. A 178 223-239. · doi:10.1111/rssa.12056 [19] Lin, D. Y., Psaty, B. M. and Kronmal, R. A. (1998). Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics 54 948-963. · Zbl 1058.62625 · doi:10.2307/2533848 [20] Liu, W., Kuramoto, S. J. and Stuart, E. A. (2013). An introduction to sensitivity analysis for unobserved confounding in nonexperimental prevention research. Prev. Sci. 14 570-580. [21] Lorch, S. A., Baiocchi, M., Ahlberg, C. E. and Small, D. S. (2012). The differential impact of delivery hospital on the outcomes of premature infants. Pediatrics 130 270-278. [22] Lu, B., Zanutto, E., Hornik, R. and Rosenbaum, P. R. (2001). Matching with doses in an observational study of a media campaign against drug abuse. J. Amer. Statist. Assoc. 96 1245-1253. · Zbl 1051.62113 · doi:10.1198/016214501753381896 [23] McCandless, L. C., Gustafson, P. and Levy, A. (2007). Bayesian sensitivity analysis for unmeasured confounding in observational studies. Stat. Med. 26 2331-2347. · doi:10.1002/sim.2711 [24] Nagler, J. (1991). The effect of registration laws and education on united-states voter turnout. Am. Polit. Sci. Rev. 85 1393-1405. [25] Robins, J. M., Rotnitzky, A. and Scharfstein, D. O. (2000). Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models. In Statistical Models in Epidemiology , the Environment , and Clinical Trials ( Minneapolis , MN , 1997) (E. Halloran and D. Berry, eds.). IMA Vol. Math. Appl. 116 1-94. Springer, New York. · Zbl 0998.62091 · doi:10.1007/978-1-4612-1284-3_1 [26] Rosenbaum, P. R. (1984). The consequences of adjusting for a concomitant variable that has been affected by the treatment. J. Roy. Statist. Soc. Ser. A 147 656-666. [27] Rosenbaum, P. R. (1996). Identification of causal effects using instrumental variables: Comment. J. Amer. Statist. Assoc. 91 465-468. [28] Rosenbaum, P. R. (1999). Using quantile averages in matched observational studies. J. R. Stat. Soc. Ser. C. Appl. Stat. 48 63-78. · Zbl 1054.62624 · doi:10.1111/1467-9876.00140 [29] Rosenbaum, P. R. (2002a). Observational Studies , 2nd ed. Springer, New York. · Zbl 0985.62091 [30] Rosenbaum, P. R. (2002b). Covariance adjustment in randomized experiments and observational studies. Statist. Sci. 17 286-327. · Zbl 1013.62117 · doi:10.1214/ss/1042727942 [31] Rosenbaum, P. R. (2005). Heterogeneity and causality: Unit heterogeneity and design sensitivity in observational studies. Amer. Statist. 59 147-152. · doi:10.1198/000313005X42831 [32] Rosenbaum, P. R. (2007). Sensitivity analysis for \(m\)-estimates, tests, and confidence intervals in matched observational studies. Biometrics 63 456-464. · Zbl 1140.62084 · doi:10.1111/j.1541-0420.2006.00717.x [33] Rosenbaum, P. R. (2010). Design of Observational Studies . Springer, New York. · Zbl 1308.62005 · doi:10.1007/978-1-4419-1213-8 [34] Rosenzweig, M. R. and Wolpin, K. I. (2000). Natural “natural experiments” in economics. J. Econ. Lit. 38 827-874. [35] Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 6 688-701. [36] Rubin, D. B. (1986). Which ifs have causal answers. J. Amer. Statist. Assoc. 81 961-962. [37] Small, D. S. (2007). Sensitivity analysis for instrumental variables regression with overidentifying restrictions. J. Amer. Statist. Assoc. 102 1049-1058. · Zbl 1333.62295 · doi:10.1198/016214507000000608 [38] Small, D. S. and Rosenbaum, P. R. (2008). War and wages: The strength of instrumental variables and their sensitivity to unobserved biases. J. Amer. Statist. Assoc. 103 924-933. · Zbl 1205.62221 · doi:10.1198/016214507000001247 [39] Sobel, M. E. (2006). What do randomized studies of housing mobility demonstrate?: Causal inference in the face of interference. J. Amer. Statist. Assoc. 101 1398-1407. · Zbl 1171.62365 · doi:10.1198/016214506000000636 [40] Splawa-Neyman, J. (1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statist. Sci. 5 465-472. · Zbl 0955.01560 · doi:10.1214/ss/1177012031 [41] Staiger, D. and Stock, J. H. (1997). Instrumental variables regression with weak instruments. Econometrica 65 557-586. · Zbl 0871.62101 · doi:10.2307/2171753 [42] Stock, J. H. and Yogo, M. (2005). Testing for weak instruments in linear IV regression. In Identification and Inference in Econometric Models : Essays in Honor of Thomas J. Rothenberg (D. W. K. Andrews and J. H. Stock, eds.) Cambridge Univ. Press, Cambridge. · Zbl 1121.62066 · doi:10.1017/CBO9780511614491.006 [43] Wald, A. (1940). The fitting of straight lines if both variables are subject to error. Ann. Math. Stat. 11 285-300. · Zbl 0023.34402 · doi:10.1214/aoms/1177731868 [44] Wolfinger, R. E. and Rosenstone, S. J. (1980). Who Votes? Yale Univ. Press, New Haven. [45] Zigler, C. M., Dominici, F. and Wang, Y. (2012). Estimating causal effects of air quality regulations using principal stratification for spatially correlated multivariate intermediate outcomes. Biostatistics 13 289-302. [46] Zubizarreta, J. R., Small, D. S. and Rosenbaum, P. R. (2014). Isolation in the construction of natural experiments. Ann. Appl. Stat. 8 2096-2121. · Zbl 1454.62432 · doi:10.1214/14-AOAS770 [47] Zubizarreta, J. R., Small, D. S., Goyal, N. K., Lorch, S. and Rosenbaum, P. R. (2013). Stronger instruments via integer programming in an observational study of late preterm birth outcomes. Ann. Appl. Stat. 7 25-50. · Zbl 1454.62431 · doi:10.1214/12-AOAS582 This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.