×
Owen, Art B.; Varian, Hal

Optimizing the tie-breaker regression discontinuity design. (English) Zbl 1452.62583

Electron. J. Stat. 14, No. 2, 4004-4027 (2020).
Summary: Motivated by customer loyalty plans and scholarship programs, we study tie-breaker designs which are hybrids of randomized controlled trials (RCTs) and regression discontinuity designs (RDDs). We quantify the statistical efficiency of a tie-breaker design in which a proportion \(\Delta\) of observed subjects are in the RCT. In a two line regression, statistical efficiency increases monotonically with \(\Delta \), so efficiency is maximized by an RCT. We point to additional advantages of tie-breakers versus RDD: for a nonparametric regression the boundary bias is much less severe and for quadratic regression, the variance is greatly reduced. For a two line model we can quantify the short term value of the treatment allocation and this comparison favors smaller \(\Delta\) with the RDD being best. We solve for the optimal tradeoff between these exploration and exploitation goals. The usual tie-breaker design applies an RCT on the middle \(\Delta\) subjects as ranked by the assignment variable. We quantify the efficiency of other designs such as experimenting only in the second decile from the top. We also show that in some general parametric models a Monte Carlo evaluation can be replaced by matrix algebra.

Cited in 2 Documents

MSC:

62K05 Optimal statistical designs
62J02 General nonlinear regression
62F10 Point estimation
62P20 Applications of statistics to economics

Keywords:

electronic commerce; experimental design; hybrid experiments
Cite Review PDF
Full Text: DOI arXiv Euclid

References:

[1] Atila Abdulkadiroglu, Joshua D. Angrist, Yusuke Narita, and Parag A. Pathak. Impact evaluation in matching markets with general tie-breaking. Technical report, National Bureau of Economic Research, 2017. URL, http://www.nber.org/papers/w24172. · Zbl 1420.91355 · doi:10.3982/ECTA13925
[2] Leona S. Aiken, Stephen G. West, David E. Schwalm, James L. Carroll, and Shenghwa Hsiung. Comparison of a randomized and two quasi-experimental designs in a single outcome evaluation: Efficacy of a university-level remedial writing program., Evaluation Review, 22(2):207-244, 1998.
[3] Joshua Angrist, Sally Hudson, and Amanda Pallais. Leveling up: Early results from a randomized evaluation of post-secondary aid. Technical report, National Bureau of Economic Research, 2014. URL, http://www.nber.org/papers/w20800.pdf.
[4] Joshua D. Angrist and Jorn-Steffen Pischke., Mostly Harmless Econometrics. Princeton Univerity Press, Princeton, 2009. · Zbl 1159.62090
[5] Joshua D. Angrist and Jorn-Steffen Pischke., Mastering Metrics. Princeton Univerity Press, Princeton, 2014. · Zbl 1360.62001
[6] Timothy B Armstrong and Michal Kolesár. Optimal inference in a class of regression models., Econometrica, 86(2):655-683, 2018. · Zbl 1414.62150 · doi:10.3982/ECTA14434
[7] George E. P. Box, William Gordon Hunter, and J. Stuart Hunter., Statistics for Experimenters. John Wiley and Sons, New York, 1978. · Zbl 0394.62003
[8] Sebastian Calonico, Matias D. Cattaneo, and Rocio Titiunik. Robust nonparametric confidence intervals for regression-discontinuity designs., Econometrica, 82(6) :2295-2326, 2014. · Zbl 1410.62066 · doi:10.3982/ECTA11757
[9] Donald T. Campbell. Reforms as experiments., American Psychologist, 24(4):409, 1969.
[10] Joseph C. Cappelleri and William M. K. Trochim. Cutoff designs. In Marcel Dekker, editor, Encyclopedia of Biopharmaceutical Statistics. CRC Press, 2003.. URL https://www.socialresearchmethods.net/research/Cutoff
[11] Andrew Gelman and Guido Imbens. Why high-order polynomials should not be used in regression discontinuity designs., Journal of Business & Economic Statistics, 2017. URL http://www.nber.org/papers/w20405.
[12] A. S. Goldberger. Selection bias in evaluating treatment effects: Some formal illustrations. Technical Report Discussion paper 128-72, Institute for Research on Poverty, University of Wisconsin-Madison, 1972. · Zbl 1190.62191
[13] Jinyong Hahn, Petra Todd, and Wilbert Van der Klaauw. Identification and estimation of treatment effects with a regression-discontinuity design., Econometrica, 69(1):201-209, 2001.
[14] Guido Imbens and Thomas Lemieux. Regression discontinuity designs: a guide to practice., Journal of Econometrics, 142(2):615-635, 2008. URL www.nber.org/papers/w13039.pdf. · Zbl 1418.62475 · doi:10.1016/j.jeconom.2007.05.001
[15] Guido Imbens and Stefan Wager. Optimized regression discontinuity designs., Review of Economics and Statistics, 101(2):264-278, 2019.
[16] Guido W. Imbens and Donald B. Rubin., Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge University Press, 2015. · Zbl 1355.62002
[17] Robin Tepper Jacob, Pei Zhu, Marie-Andrée, Somers, and Howard Bloom. A practical guide to regression discontinuity., MDRC Publications, July 2012. URL https://www.mdrc.org/publication/practical-guide-regression-discontinuity.
[18] David S. Lee and Thomas Lemieux. Regression discontinuity designs in economics., Journal of Economic Literature, 48:281-355, June 2010. URL https://www.princeton.edu/ davidlee/wp/RDDEconomics.pdf.
[19] Justin McCrary. Manipulation of the running variable in the regression discontinuity design: A density test., Journal of Econometrics, 142(2):698-714, 2008. · Zbl 1418.62142 · doi:10.1016/j.jeconom.2007.05.005
[20] Jack Porter. Estimation in the regression discontinuity model., Unpublished Manuscript, Department of Economics, University of Wisconsin at Madison, 2003:5-19, 2003.
[21] John Rice and Murray Rosenblatt. Smoothing splines: regression, derivatives and deconvolution., The Annals of Statistics, pages 141-156, 1983. · Zbl 0535.41019 · doi:10.1214/aos/1176346065
[22] Evan Rosenman and Karthik Rajkumar. Optimized partial identification bounds for regression discontinuity designs with manipulation. Technical Report, arXiv:1910.02170, Stanford University, 2019.
[23] Steven L. Scott. Multi-armed bandit experiments in the online service economy., Applied Stochastic Models in Business and Industry, 31(1):37-45, 2015. · Zbl 07883193
[24] Student. The Lanarkshire milk experiment., Biometrika, 23(2/3):398-406, 1931. · JFM 57.0632.05 · doi:10.1093/biomet/23.3-4.407
[25] D. L. Thistlethwaite and D. T. Campbell. Regression-discontinuity analysis: An alternative to the ex post facto experiment., Journal of Educational Psychology, 51(6):309, 1960.
[26] Wilbert Van Der Klaauw. Regression-discontinuity analysis: A survey of recent developments in economics., LABOUR, 22(2):219-245, 2008. URL http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.466.956&rep=rep1&type=pdf. · Zbl 1418.62539 · doi:10.1016/j.jeconom.2007.05.007
[27] C. · Zbl 1229.62100
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.