×

zbMATH — the first resource for mathematics

Rerandomization to improve covariate balance in experiments. (English) Zbl 1274.62509
Summary: Randomized experiments are the “gold standard” for estimating causal effects, yet often in practice, chance imbalances exist in covariate distributions between treatment groups. If covariate data are available before units are exposed to treatments, these chance imbalances can be mitigated by first checking covariate balance before the physical experiment takes place. Provided a precise definition of imbalance has been specified in advance, unbalanced randomizations can be discarded, followed by a rerandomization, and this process can continue until a randomization yielding balance according to the definition is achieved. By improving covariate balance, rerandomization provides more precise and trustworthy estimates of treatment effects.

MSC:
62K99 Design of statistical experiments
PDF BibTeX XML Cite
Full Text: DOI Euclid arXiv
References:
[1] Aickin, M. (2001). Randomization, balance, and the validity and efficiency of design-adaptive allocation methods. J. Statist. Plann. Inference 94 97-119. · Zbl 0976.62099 · doi:10.1016/S0378-3758(00)00228-7
[2] Anscombe, F. J. (1948a). The validity of comparative experiments. J. Roy. Statist. Soc. Ser. A. 111 181-211. · Zbl 0032.03702 · doi:10.2307/2984159
[3] Arnold, G. C. (1986). Randomization: A historic controversy. In The Fascination of Statistics (R. J. Brook, G. C. Arnold, T. H. Hassard and R. M. Pringle, eds.) 231-244. CRC Press, Boca Raton, FL.
[4] Atkinson, A. C. (2002). The comparison of designs for sequential clinical trials with covariate information. J. Roy. Statist. Soc. Ser. A 165 349-373. · Zbl 1001.62522 · doi:10.1111/1467-985X.00564
[5] Bailey, R. A. (1983). Restricted randomization. Biometrika 70 183-198. · Zbl 0517.62069 · doi:10.1093/biomet/70.1.183
[6] Bailey, R. A. (1986). Randomization, constrained. Encyclopedia of Statistical Sciences 7 519-524.
[7] Bailey, R. A. and Rowley, C. A. (1987). Valid randomization. Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 410 105-124. · Zbl 0612.62110 · doi:10.1098/rspa.1987.0030
[8] Birkett, N. J. (1985). Adaptive allocation in randomized controlled trials. Control Clin Trials 6 146-155.
[9] Brillinger, D., Jones, L. and Tukey, J. (1978). The Management of Weather Resources II : The Role of Statistics in Weather Resources Management . US Government Printing Office, Washington, DC.
[10] Bruhn, M. and McKenzie, D. (2009). In pursuit of balance: Randomization in practice in development field experiments. American Economic Journal : Applied Economics 1 200-232.
[11] Cochran, W. G. and Rubin, D. B. (1973). Controlling bias in observational studies: A review. Sankhyā Ser. A 35 417-446. · Zbl 0291.62012
[12] Cox, D. R. (1982). Randomization and concomitant variables in the design of experiments. In Statistics and Probability : Essays in Honor of C. R. Rao 197-202. North-Holland, Amsterdam. · Zbl 0482.62067
[13] Cox, D. R. (2009). Randomization in the Design of Experiments. International Statistical Review 77 415-429.
[14] Efron, B. (1971). Forcing a sequential experiment to be balanced. Biometrika 58 403-417. · Zbl 0226.62086 · doi:10.1093/biomet/58.3.403
[15] Erdős, P. and Rényi, A. (1959). On the central limit theorem for samples from a finite population. Magyar Tud. Akad. Mat. Kutató Int. Közl. 4 49-61. · Zbl 0086.34001
[16] Fisher, R. A. (1926). The arrangement of field experiments. Journal of the Ministry of Agriculture of Great Britain 33 503-513.
[17] Fisher, R. A. (1935). The Design of Experiments . Oliver and Boyd, Edinburgh. · Zbl 0011.03205
[18] Freedman, D. A. (2008). On regression adjustments to experimental data. Adv. in Appl. Math. 40 180-193. · Zbl 1130.62003 · doi:10.1016/j.aam.2006.12.003
[19] Garthwaite, P. H. (1996). Confidence intervals from randomization tests. Biometrics 1387-1393. · Zbl 0925.62122 · doi:10.2307/2532852
[20] Gosset, W. J. (1938). Comparison between balanced and random arrangements of field plots. Biometrika 29 363. · Zbl 0018.03504 · doi:10.1093/biomet/29.3-4.363
[21] Greenberg, B. G. (1951). Why randomize? Biometrics 7 309-322.
[22] Greevy, R., Lu, B., Silber, J. H. and Rosenbaum, P. (2004). Optimal multivariate matching before randomization. Biostatistics 5 263-275. · Zbl 1096.62078 · doi:10.1093/biostatistics/5.2.263
[23] Grundy, P. M. and Healy, M. J. R. (1950). Restricted randomization and quasi-Latin squares. J. R. Stat. Soc. Ser. B Stat. Methodol. 12 286-291. · Zbl 0041.45809
[24] Hájek, J. (1960). Limiting distributions in simple random sampling from a finite population. Magyar Tud. Akad. Mat. Kutató Int. Közl. 5 361-374. · Zbl 0102.15001
[25] Hansen, B. B. and Bowers, J. (2008). Covariate balance in simple, stratified and clustered comparative studies. Statist. Sci. 23 219-236. · Zbl 1327.62019 · doi:10.1214/08-STS254
[26] Harville, D. A. (1975). Experimental randomization: Who needs it? Amer. Statist. 27-31. · Zbl 0361.62071 · doi:10.2307/2683676
[27] Ho, D. E., Imai, K., King, G. and Stuart, E. A. (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 15 199-236.
[28] Holschuh, N. (1980). Randomization and design: I. In R. A. Fisher : An Appreciation (S. E. Fienberg and D. V. Hinkley, eds.). Lecture Notes in Statistics 1 35-45. Springer, New York.
[29] Imai, K., King, G. and Stuart, E. A. (2008). Misunderstanding between experimentalists and observationalists about causal inference. J. Roy. Statist. Soc. Ser. A 171 481-502. · Zbl 05529657 · doi:10.1111/j.1467-985X.2007.00527.x
[30] Imai, K., King, G. and Nall, C. (2009). The essential role of pair matching in cluster-randomized experiments, with application to the Mexican universal health insurance evaluation. Statist. Sci. 24 29-53. · Zbl 1327.62061 · doi:10.1214/08-STS274
[31] Keele, L., McConnaughy, C., White, I., List, P. M. E. M. and Bailey, D. (2009). Adjusting experimental data. In Experiments in Political Science Conference .
[32] Kempthorne, O. (1955). The randomization theory of experimental inference. J. Amer. Statist. Assoc. 50 946-967.
[33] Kempthorne, O. (1986). Randomization II. Encyclopedia of Statistical Sciences 7 519-524.
[34] Krause, M. S. and Howard, K. I. (2003). What random assignment does and does not do. Journal of Clinical Psychology 59 751-766.
[35] Lehmann, E. L. and Romano, J. P. (2005). Testing Statistical Hypotheses , 3rd ed. Springer Texts in Statistics . Springer, New York. · Zbl 1076.62018
[36] Lock, K. F. (2011). Rerandomization to improve covariate balance in randomized experiments Ph.D. thesis, Harvard Univ., Cambridge, MA.
[37] Maclure, M., Nguyen, A., Carney, G., Dormuth, C., Roelants, H., Ho, K. and Schneeweiss, S. (2006). Measuring prescribing improvements in pragmatic trials of educational tools for general practitioners. Basic & Clinical Pharmacology & Toxicology 98 243-252.
[38] Manly, B. F. J. (2007). Randomization , Bootstrap and Monte Carlo Methods in Biology , 3rd ed. Chapman & Hall/CRC, Boca Raton, FL. · Zbl 1269.62076
[39] Mardia, K. V., Kent, J. T. and Bibby, J. M. (1980). Multivariate Analysis . Academic Press, London. · Zbl 0432.62029
[40] McEntegart, D. J. (2003). The pursuit of balance using stratified and dynamic randomization techniques: An overview. Drug Information Journal 37 293-308.
[41] Morris, C. (1979). A finite selection model for experimental design of the health insurance study. J. Econometrics 11 43-61. · Zbl 0413.62084 · doi:10.1016/0304-4076(79)90053-8
[42] Morris, C. N. and Hill, J. L. (2000). The health insurance experiment: Design using the finite selection model. In Public Policy and Statistics : Case Studies from RAND 29-53. Springer, New York.
[43] Moulton, L. H. (2004). Covariate-based constrained randomization of group-randomized trials. Clin Trials 1 297-305.
[44] Pocock, S. J. (1979). Allocation of patients to treatment in clinical trials. Biometrics 35 183-197.
[45] Pocock, S. J. and Simon, R. (1975). Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics 31 103-115.
[46] Raynor, A. A. (1986). Some Sidelights on Experimental Design. In The Fascination of Statistics (R. J. Brook, G. C. Arnold, T. H. Hassard and R. M. Pringle, eds.) 245-264. CRC Press, Boca Raton, FL.
[47] Rosenberger, W. F. and Lachin, J. M. (2002). Randomization in Clinical Trials : Theory and Practice . Wiley, New York. · Zbl 1007.62091
[48] Rosenberger, W. F. and Sverdlov, O. (2008). Handling covariates in the design of clinical trials. Statist. Sci. 23 404-419. · Zbl 1329.62350 · doi:10.1214/08-STS269
[49] Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 66 688.
[50] Rubin, D. B. (1976). Multivariate matching methods that are equal percent bias reducing. I. Some examples. Biometrics 32 109-120. · Zbl 0326.62043 · doi:10.2307/2529342
[51] Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Ann. Statist. 6 34-58. · Zbl 0383.62021 · doi:10.1214/aos/1176344064
[52] Rubin, D. B. (1980). Randomization analysis of experimental data: The Fisher randomization test comment. J. Amer. Statist. Assoc. 75 591-593. · Zbl 0444.62089 · doi:10.2307/2287648
[53] Rubin, D. B. (2006). Matched Sampling for Causal Effects . Cambridge Univ. Press, Cambridge. · Zbl 1118.62113 · doi:10.1017/CBO9780511810725
[54] Rubin, D. B. (2008a). Comment: The design and analysis of gold standard randomized experiments. J. Amer. Statist. Assoc. 103 1350-1353. · Zbl 1286.62012 · doi:10.1198/016214508000001011
[55] Rubin, D. B. (2008b). For objective causal inference, design trumps analysis. Ann. Appl. Stat. 2 808-804. · Zbl 1149.62089 · doi:10.1214/08-AOAS187
[56] Rubin, D. B. and Thomas, N. (1992). Affinely invariant matching methods with ellipsoidal distributions. Ann. Statist. 20 1079-1093. · Zbl 0761.62065 · doi:10.1214/aos/1176348671
[57] Savage, L. J. (1962). The Foundations of Statistical Inference . Methuen & Co. Ltd., London.
[58] Scott, N. W., McPherson, G. C., Ramsay, C. R. and Campbell, M. K. (2002). The method of minimization for allocation to clinical trials. a review. Control Clinical Trials 23 662-674.
[59] Seidenfeld, T. (1981). Levi on the dogma of randomization in experiments. In Henry E. Kyburg , Jr. & Isaac Levi (R. J. Bogdan, ed.) 263-291. Springer, Berlin.
[60] Simon, R. (1979). Restricted randomization designs in clinical trials. Biometrics 35 503-512.
[61] Soares, J. F. and Wu, C. F. J. (1985). Optimality of random allocation design for the control of accidental bias in sequential experiments. J. Statist. Plann. Inference 11 81-87. · Zbl 0574.62068 · doi:10.1016/0378-3758(85)90027-8
[62] Splawa-Neyman, J. (1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statist. Sci. 5 465-472. · Zbl 0955.01560
[63] Sprott, D. A. and Farewell, V. T. (1993). Randomization in experimental science. Statist. Papers 34 89-94. · doi:10.1007/BF02925530
[64] Tukey, J. W. (1993). Tightening the clinical trial. Control Clin Trials 14 266-285.
[65] Urbach, P. (1985). Randomization and the design of experiments. Philos. Sci. 52 256-273. · doi:10.1086/289243
[66] White, S. J. and Freedman, L. S. (1978). Allocation of patients to treatment groups in a controlled clinical study. British Journal of Cancer 37 849.
[67] Worrall, J. (2010). Evidence: Philosophy of science meets medicine. J. Eval. Clin. Pract. 16 356-362.
[68] Xu, Z. and Kalbfleisch, J. D. (2010). Propensity score matching in randomized clinical trials. Biometrics 66 813-823. · Zbl 1203.62208 · doi:10.1111/j.1541-0420.2009.01364.x
[69] Yates, F. (1939). The comparative advantages of systematic and randomized arrangements in the design of agricultural and biological experiments. Biometrika 30 440. · JFM 65.0591.03
[70] Yates, F. (1948). Contribution to the discussion of “The validity of comparative experiments” by FJ Anscombe. J. Roy. Statist. Soc. Ser. A 111 204-205. · doi:10.2307/2984159
[71] Youden, W. J. (1972). Randomization and experimentation. Technometrics 14 13-22.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.