A deep learning semiparametric regression for adjusting complex confounding structures. (English) Zbl 1478.62284

Summary: Deep Treatment Learning (deepTL), a robust yet efficient deep learning-based semiparametric regression approach, is proposed to adjust the complex confounding structures in comparative effectiveness analysis of observational data, for example, electronic health record (EHR) data in which complex confounding structures are often embedded. Specifically, we develop a deep learning neural network with a score-based ensembling scheme for flexible function approximation. An improved semiparametric procedure is further developed to enhance the performance of the proposed method under finite sample settings. Comprehensive numerical studies have demonstrated the superior performance of the proposed methods, as compared with existing methods, with a remarkably reduced bias and mean squared error in parameter estimates. The proposed research is motivated by a postsurgery pain study, which is also used to illustrate the practical application of deepTL. Finally, an R package, “deepTL”, is developed to implement the proposed method.


62M45 Neural nets and related approaches to inference from stochastic processes
62G08 Nonparametric regression and quantile regression
62G09 Nonparametric statistical resampling methods
62P10 Applications of statistics to biology and medical sciences; meta analysis
68T07 Artificial neural networks and deep learning


R; deepTL
Full Text: DOI


[1] Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar. Behav. Res. 46 399-424.
[2] Bengio, Y., Delalleau, O. and Roux, N. L. (2006). The curse of highly variable functions for local kernel machines. In Advances in Neural Information Processing Systems 107-114.
[3] Benson, K. and Hartz, A. J. (2000). A comparison of observational studies and randomized, controlled trials. N. Engl. J. Med. 342 1878-1886.
[4] Breiman, L. (1996). Bagging predictors. Mach. Learn. 24 123-140. · Zbl 0858.68080
[5] Britton, A., McKee, M., Black, N., McPherson, K., Sanderson, C. and Bain, C. (1997). Choosing between randomised and non-randomised studies: A systematic review. Health Technology Assessment (Winchester, England) 2 i-iv.
[6] Byrd, R. H., Chin, G. M., Nocedal, J. and Wu, Y. (2012). Sample size selection in optimization methods for machine learning. Math. Program. 134 127-155. · Zbl 1252.49044
[7] Chen, Y., Carroll, R. J., Hinz, E. R. M., Shah, A., Eyler, A. E., Denny, J. C. and Xu, H. (2013). Applying active learning to high-throughput phenotyping algorithms for electronic health records data. J. Am. Med. Inform. Assoc. 20 e253-e259.
[8] Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C. and Newey, W. K. (2016). Double machine learning for treatment and causal parameters. Technical Report No. CWP49/16. CeMMAP working paper, Centre for Microdata Methods and Practice, London.
[9] Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W. and Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. Econom. J. 21 C1-C68.
[10] Curry, J. I., Reeves, B. and Stringer, M. D. (2003). Randomized controlled trials in pediatric surgery: Could we do better? Journal of Pediatric Surgery 38 556-559.
[11] Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Math. Control Signals Systems 2 303-314. · Zbl 0679.94019
[12] Engle, R. F., Granger, C. W., Rice, J. and Weiss, A. (1986). Semiparametric estimates of the relation between weather and electricity sales. J. Amer. Statist. Assoc. 81 310-320.
[13] Faragó, A. and Lugosi, G. (1993). Strong universal consistency of neural network classifiers. IEEE Trans. Inf. Theory 39 1146-1151. · Zbl 0811.68109
[14] Hansen, L. K. and Salamon, P. (1990). Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12 993-1001.
[15] Hornik, K., Stinchcombe, M. and White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Netw. 2 359-366. · Zbl 1383.92015
[16] Kazley, A. S. and Ozcan, Y. A. (2008). Do hospitals with electronic medical records (EMRs) provide higher quality care?: An examination of three clinical conditions. Med. Care Res. Rev. 65 496-513.
[17] Kinga, D. and Adam, J. B. (2015). A method for stochastic optimization. In International Conference on Learning Representations (ICLR) 5. · Zbl 1342.30016
[18] LeCun, Y., Bengio, Y. and Hinton, G. (2015). Deep learning. Nature 521 436-444.
[19] MacLehose, R., Reeves, B., Harvey, I., Sheldon, T., Russell, I. and Black, A. (2000). A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technology Assessment (Winchester, England) 4 1-154.
[20] McCulloch, P., Taylor, I., Sasako, M., Lovett, B. and Griffin, D. (2002). Randomised trials in surgery: Problems and possible solutions. BMJ, Br. Med. J. 324 1448-1451.
[21] Mi, X., Tighe, P., Zou, F. and Zou, B. (2021). Supplement to “A deep learning semiparametric regression for adjusting complex confounding structures.” https://doi.org/10.1214/21-AOAS1481SUPPA, https://doi.org/10.1214/21-AOAS1481SUPPB
[22] Mi, X., Zou, F. and Zhu, R. (2019). Bagging and deep learning in optimal individualized treatment rules. Biometrics 75 674-684. · Zbl 1436.62609
[23] Miriovsky, B. J., Shulman, L. N. and Abernethy, A. P. (2012). Importance of health information technology, electronic health records, and continuously aggregating data to comparative effectiveness research and learning health care. J. Clin. Oncol. 30 4243-4248.
[24] Murdoch, T. B. and Detsky, A. S. (2013). The inevitable application of big data to health care. JAMA 309 1351-1352.
[25] Nie, X. and Wager, S. (2017). Learning objectives for treatment effect estimation. Preprint. Available at arXiv:1712.04912.
[26] Platt, J. et al. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers 10 61-74.
[27] Psaty, B. M. and Larson, E. B. (2013). Investments in infrastructure for diverse research resources and the health of the public. JAMA 309 1895-1896.
[28] Robins, J. M., Rotnitzky, A. and Zhao, L. P. (1994). Estimation of regression coefficients when some regressors are not always observed. J. Amer. Statist. Assoc. 89 846-866. · Zbl 0815.62043
[29] Robinson, P. M. (1988). Root-\(N\)-consistent semiparametric regression. Econometrica 56 931-954. · Zbl 0647.62100
[30] Rosenbaum, P. R. and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika 70 41-55. · Zbl 0522.62091
[31] Schisterman, E. F., Cole, S. R. and Platt, R. W. (2009). Overadjustment bias and unnecessary adjustment in epidemiologic studies. Epidemiology (Cambridge, Mass.) 20 488-495.
[32] Shah, N. H. and Tenenbaum, J. D. (2012). Focus on translational bioinformatics: The coming age of data-driven medicine: Translational bioinformatics’ next frontier. J. Am. Med. Inform. Assoc. 19 e2.
[33] Shir, Y., Raja, S. N. and Frank, S. M. (1994). The effect of epidural versus general anesthesia on postoperative pain and analgesic requirements in patients undergoing radical prostatectomy. Anesthesiology 80 49-56.
[34] Sonoda, S. and Murata, N. (2017). Neural network with unbounded activation functions is universal approximator. Appl. Comput. Harmon. Anal. 43 233-268. · Zbl 1420.68177
[35] Stock, J. H. (1991). Nonparametric policy analysis: An application to estimating hazardous waste cleanup benefits. In Nonparametric and Semiparametric Methods in Econometrics and Statistics 77-98. Cambridge University Press, Cambridge.
[36] Tighe, P. J., King, C. D., Zou, B. and Fillingim, R. B. (2016). Time to onset of sustained postoperative pain relief (SuPPR): Evaluation of a new systems-level metric for acute pain management. The Clinical Journal of Pain 32 371-379.
[37] Tipping, M. E. (2001). Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 1 211-244. · Zbl 0997.68109
[38] Toh, S., García Rodríguez, L. A. and Hernán, M. A. (2011). Confounding adjustment via a semi-automated high-dimensional propensity score algorithm: An application to electronic medical records. Pharmacoepidemiol. Drug Saf. 20 849-857.
[39] Tu, J. V. (1996). Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J. Clin. Epidemiol. 49 1225-1231.
[40] Tverskoy, M., Cozacov, C., Ayache, M., Bradley, J. E. and Kissin, I. (1990). Postoperative pain after inguinal herniorrhaphy with different types of anesthesia. Anesthesia and Analgesia 70 29-35.
[41] Williams, C. K. and Barber, D. (1998). Bayesian classification with Gaussian processes. IEEE Trans. Pattern Anal. Mach. Intell. 20 1342-1351.
[42] Zhou, Z.-H., Wu, J. and Tang, W. (2002). Ensembling neural networks: Many could be better than all. Artificial Intelligence 137 239-263. · Zbl 0995.68077
[43] Zou, B., Zou, F., Shuster, J. J., Tighe, P. J., Koch, G. G. and Zhou, H. (2016). On variance estimate for covariate adjustment by propensity score analysis. Stat. Med. 35 3537-3548
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.