Evaluating the causal effect of university grants on student dropout: evidence from a regression discontinuity design using principal stratification. (English) Zbl 1397.62585

Summary: Regression discontinuity (RD) designs are often interpreted as locally randomized experiments for units with a realized value of a pretreatment variable falling around a threshold. Motivated by the evaluation of Italian university grants, we consider a fuzzy RD design where the treatment status is based on both eligibility criteria and a voluntary application status. Resting on the fact that grant application and grant receipt statuses are post-assignment (post-eligibility) intermediate variables, we use the principal stratification framework to define causal estimands within the Rubin Causal Model. We propose a probabilistic formulation of the assignment mechanism underlying RD designs, by reformulating the Stable Unit Treatment Value Assumption (SUTVA) and making an explicit local overlap assumption for a subpopulation around the threshold. We invoke a local randomization assumption instead of the more standard continuity assumptions. We also develop a Bayesian approach to select the target subpopulation(s) with adjustment for multiple comparisons, and to draw inference for the target causal estimands within this framework. Applying the method to the data from two Italian universities, we find evidence that university grants are effective in preventing students from low-income families from dropping out of higher education.


62P25 Applications of statistics to social sciences
62F15 Bayesian inference
Full Text: DOI arXiv Euclid


[1] Albert, J. H. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. J. Amer. Statist. Assoc. 88 669-679. · Zbl 0774.62031
[2] Angrist, J. D., Imbens, G. W. and Rubin, D. B. (1996). Identification of causal effects using instrumental variables (with discussion). J. Amer. Statist. Assoc. 91 444-472. · Zbl 0897.62130
[3] Battistin, E. and Rettore, E. (2008). Ineligibles and eligible non-participants as a double comparison group in regression discontinuity designs. J. Econometrics 142 715-730. · Zbl 1418.62416
[4] Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300. · Zbl 0809.62014
[5] Berk, R. A. and de Leuuw, J. (1999). An evaluation of California’s inmate classification system using a generalized regression discontinuity design. J. Amer. Statist. Assoc. 94 1045-1052.
[6] Berry, S. M. and Berry, D. A. (2004). Accounting for multiplicities in assessing drug safety: A three-level hierarchical mixture model. Biometrics 60 418-426. · Zbl 1125.62118
[7] Cattaneo, M. D., Frandsen, B. and Titiunik, R. (2015). Randomization inference in the regression discontinuity design: An application to party advantages in the U.S. Senate. Journal of Causal Inference 3 1-24.
[8] Cellini, S. R., Ferreira, F. and Rothstein, J. (2010). The value of school facility investments: Evidence from a dynamic regression discontinuity design. Q. J. Econ. 125 215-261.
[9] Chib, S. and Greenberg, E. (2014). Nonparametric Bayes analysis of the sharp and fuzzy regression discontinuity designs. Technical report, Washington Univ. St Louis, Olin School of Business.
[10] Chib, S. and Jacobi, L. (2015). Bayesian fuzzy regression discontinuity analysis and returns to compulsory schooling. J. Appl. Econometrics . Published online in Wiley Online Library ( ), DOI:10.1002/jae.2481 .
[11] Cook, T. D. (2008). “Waiting for life to arrive”: A history of the regression-discontinuity design in psychology, statistics and economics. J. Econometrics 142 636-654. · Zbl 1418.62436
[12] de Finetti, B. (1937). La prévision: ses lois logiques, ses sources subjectives. Ann. Inst. Henri Poincaré 7 1-68. · Zbl 0017.07602
[13] Dinardo, J. and Lee, D. S. (2011). Program evaluation and research designs. In Handbook of Labor Economics 4A 463-536. Elsevier, Philadelphia, PA.
[14] Elliott, M. R., Raghunathan, T. E. and Li, Y. (2010). Bayesian inference for causal mediation effects using principal stratification with dichotomous mediators and outcomes. Biostatistics 11 353-372.
[15] Frangakis, C. E. and Rubin, D. B. (2002). Principal stratification in causal inference. Biometrics 58 21-29. · Zbl 1209.62288
[16] Frumento, P., Mealli, F., Pacini, B. and Rubin, D. B. (2012). Evaluating the effect of training on wages in the presence of noncompliance, nonemployment, and missing outcome data. J. Amer. Statist. Assoc. 107 450-466. · Zbl 1328.62609
[17] Garibaldi, P., Giavazzi, F., Ichino, A. and Rettore, E. (2012). College cost and time to complete a degree: Evidence from tuition discontinuities. Rev. Econ. Stat. 94 699-711.
[18] Gelman, A., Meng, X.-L. and Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statist. Sinica 6 733-807. · Zbl 0859.62028
[19] Gelman, A. E. and Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statist. Sci. 7 457-472. · Zbl 1386.65060
[20] Gosselin, F. (2011). A new calibrated Bayesian internal goodness-of-fit method: Sampled posterior \(p\)-values as simple and general \(p\)-values that allow double use of the data. PLoS ONE 6 1-10.
[21] Hahn, J., Todd, P. E. and Van der Klaauw, W. (2001). Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica 69 201-209.
[22] Imbens, G. W. (2004). Nonparametric estimation of average treatment effects under exogeneity: A review. The Review of Economics and Statistics 86 4-29.
[23] Imbens, G. W. and Angrist, J. D. (1994). Identification and estimation of local average treatment effects. Econometrica 62 467-476. · Zbl 0800.90648
[24] Imbens, G. and Kalyanaraman, K. (2012). Optimal bandwidth choice for the regression discontinuity estimator. Rev. Econ. Stud. 79 933-959.
[25] Imbens, G. W. and Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. J. Econometrics 142 615-635. · Zbl 1418.62475
[26] Imbens, G. W. and Rubin, D. B. (1997). Bayesian inference for causal effects in randomized experiments with noncompliance. Ann. Statist. 25 305-327. · Zbl 0877.62005
[27] Imbens, G. W. and Rubin, D. B. (2015). Causal Inference-for Statistics , Social , and Biomedical Sciences : An Introduction . Cambridge Univ. Press, New York. · Zbl 1355.62002
[28] Imbens, G. W. and Zajonc, T. (2011). Regression discontinuity design with multiple forcing variables. Technical report, Harvard Univ., Dept. Economics.
[29] Johnson, V. E. (2007). Bayesian model assessment using pivotal quantities. Bayesian Anal. 2 719-733. · Zbl 1331.62147
[30] Lee, D. S. (2008). Randomized experiments from non-random selection in U.S. House elections. J. Econometrics 142 675-697. · Zbl 1418.62500
[31] Lee, D. S. and Lemieux, T. (2010). Regression discontinuity designs in economics. J. Econ. Lit. 485 281-355.
[32] Li, F., Mattei, A. and Mealli, F. (2015). Supplement to “Evaluating the causal effect of university grants on student dropout: Evidence from a regression discontinuity design using principal stratification.” . · Zbl 1397.62585
[33] Ludwig, J. and Miller, D. L. (2007). Does head start improve children’s life chances? Evidence from a regression discontinuity design. Q. J. Econ. 122 15981-208.
[34] Mattei, A., Li, F. and Mealli, F. (2013). Exploiting multiple outcomes in Bayesian principal stratification analysis with application to the evaluation of a job training program. Ann. Appl. Stat. 7 2336-2360. · Zbl 1283.62054
[35] Mealli, F. and Pacini, B. (2013). Using secondary outcomes to sharpen inference in randomized experiments with noncompliance. J. Amer. Statist. Assoc. 108 1120-1131. · Zbl 06224991
[36] Mealli, F. and Rampichini, C. (2012). Evaluating the effects of university grants by using regression discontinuity designs. J. Roy. Statist. Soc. Ser. A 175 775-798. · Zbl 06064212
[37] Mealli, F. and Rubin, B. D. (2002). Assumptions when analyzing randomized experiments with noncompliance and missing outcomes. Health Serv. Outcomes Res. Methodol. 3 225-232.
[38] Mercatanti, A. (2013). A likelihood-based analysis for relaxing the exclusion restriction in randomized experiments with noncompliance. Aust. N. Z. J. Stat. 55 129-153. · Zbl 1336.62076
[39] Mercatanti, A., Li, F. and Mealli, F. (2015). Improving inference of Gaussian mixtures using auxiliary variables. Stat. Anal. Data Min. 8 34-48.
[40] Murphy, S. A. (2003). Optimal dynamic treatment regimes. J. R. Stat. Soc. Ser. B Stat. Methodol. 65 331-366. · Zbl 1065.62006
[41] Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psycology 66 688-701.
[42] Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Ann. Statist. 6 34-58. · Zbl 0383.62021
[43] Rubin, D. B. (1980). Discussion of “Randomization analysis of experimental data: The Fisher randomization test” by D. Basu. J. Amer. Statist. Assoc. 75 591-593.
[44] Sales, A. and Hansen, B. (2014). Limitless regression discontinuity: Causal inference for a population surrounding a threshold. Available at . arXiv:1403.5478
[45] Schwartz, S. L., Li, F. and Mealli, F. (2011). A Bayesian semiparametric approach to intermediate variables in causal inference. J. Amer. Statist. Assoc. 31 949-962. · Zbl 1234.62022
[46] Scott, J. G. and Berger, J. O. (2006). An exploration of aspects of Bayesian multiple testing. J. Statist. Plann. Inference 136 2144-2162. · Zbl 1087.62039
[47] Thistlethwaite, D. and Campbell, D. (1960). Regression-discontinuity analysis: An alternative to the ex-post facto experiment. J. Educ. Psychol. 51 309-317.
[48] van der Klaauw, W. (2002). Estimating the effect of financial aid offers on college enrollment: A regression-discontinuity approach. Internat. Econom. Rev. 43 1249-1287.
[49] van der Klaauw, W. (2008). Regression-discontinuity analysis: A survey of recent development in economics. Labour 22 219-245.
[50] Zajonc, T. (2012). Bayesian inference for dynamic treatment regimes: Mobility, equity, and efficiency in student tracking. J. Amer. Statist. Assoc. 107 80-92. · Zbl 1261.62107
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.