Comparison and assessment of epidemic models. (English) Zbl 1407.62400

Summary: Model criticism is a growing focus of research in stochastic epidemic modelling, following the successful addressing of model fitting and parameter estimation via powerful computationally intensive statistical methods in recent decades. In this paper, we consider a variety of stochastic representations of epidemic outbreaks, with emphasis on individual-based continuous-time models, and review the range of model comparison and assessment approaches currently applied. We highlight some of the factors that can serve to impede checking and criticism of epidemic models such as lack of replication, partial observation of processes, lack of prior knowledge on parameters in competing models, the nonnested nature of models to be compared, and computational challenges. Based on a wide selection of approaches as reported in the literature, we argue that assessment and comparison of stochastic epidemic models is complex and often, by necessity, idiosyncratic to specific applications. We particularly advocate following the advice of G. E. P. Box [J. Am. Stat. Assoc. 71, 791–799 (1976; Zbl 0335.62002)] to be selective regarding the model inadequacies for which one tests and, moreover, to be open to the blending of classical and Bayesian ideas in epidemic model criticism, rather than adhering to a single philosophy.


62P10 Applications of statistics to biology and medical sciences; meta analysis
62F10 Point estimation
62F15 Bayesian inference
92D30 Epidemiology


Zbl 0335.62002
Full Text: DOI Euclid


[1] Bailey, N. T. J. (1975). The Mathematical Theory of Infectious Diseases and Its Applications, 2nd ed. Hafner Press, New York. · Zbl 0334.92024
[2] Bjørnstad, O. N. and Falck, W. (2001). Nonparametric spatial covariance functions: Estimation and testing. Environ. Ecol. Stat.8 53–70.
[3] Blum, M. and Tran, V. (2010). HIV with contact tracing: A case study in approximate Bayesian computation. Biostatistics11 644–660.
[4] Box, G. E. P. (1976). Science and statistics. J. Amer. Statist. Assoc.71 791–799. · Zbl 0335.62002
[5] Box, G. E. P. (1980). Sampling and Bayes’ inference in scientific modelling and robustness. J. Roy. Statist. Soc. Ser. A143 383–430. · Zbl 0471.62036
[6] Boys, R. J. and Giles, P. R. (2007). Bayesian inference for stochastic epidemic models with time-inhomogeneous removal rates. J. Math. Biol.55 223–247. · Zbl 1127.62107
[7] Celeux, G., Forbes, F., Robert, C. P. and Titterington, D. M. (2006). Deviance information criteria for missing data models. Bayesian Anal.1 651–673. · Zbl 1331.62329
[8] Chis Ster, I., Singh, B. K. and Ferguson, N. M. (2009). Epidemiological inference for partially observed epidemics: The example of the 2001 foot and mouth epidemic in Great Britain. Epidemics1 21–34.
[9] Clancy, D. and O’Neill, P. D. (2007). Exact Bayesian inference and model selection for stochastic models of epidemics among a community of households. Scand. J. Stat.34 259–274. · Zbl 1142.62099
[10] Cook, A. R., Gibson, G. J., Gottwald, T. and Gilligan, C. A. (2008). Constructing the effect of alternative intervention strategies in historic epidemics. J. R. Soc. Interface5 1203–1213.
[11] Dawid, A. P. and Stone, M. (1982). The functional-model basis of fiducial inference. Ann. Statist.10 1054–1074. · Zbl 0511.62010
[12] Deeth, L., Deardon, R. and Gillis, D. (2015). Model choice using the Deviance Information Criterion for latent conditional individual-level models of infectious disease spread. Epidemiologic Methods4 47–68. · Zbl 1343.92467
[13] De Angelis, D., Presanis, A. M., Birrell, P. J., Tomba, G. S. and House, T. (2015). Four key challenges in infectious disease modelling using data from multiple sources. Epidemics10 83–87.
[14] Draper, D. (1995). Assessment and propagation of model uncertainty. J. R. Stat. Soc. Ser. B. Stat. Methodol.57 45–97. · Zbl 0812.62001
[15] Finkenstädt, B. F. and Grenfell, B. T. (2000). Time series modelling of childhood diseases: A dynamical systems approach. J. R. Stat. Soc. Ser. C. Appl. Stat.49 187–205. · Zbl 0944.62100
[16] Gibson, G. J., Otten, W., Filipe, J. A. N., Cook, A., Marion, G. and Gilligan, C. A. (2006). Bayesian estimation for percolation models of disease spread in plant populations. Stat. Comput.16 391–402.
[17] Gibson, G. J. and Renshaw, E. (2001). Likelihood estimation for stochastic compartmental models using Markov chain methods. Stat. Comput.11 347–358.
[18] Gigerenzer, G. (1993). The superego, the ego, and the id in statistical reasoning. In A Handbook for Data Analysis in the Behavioral Sciences: Methodological Issues (G. Keren and C. Lewis, eds.). Lawrence Erlbaum Associates, Hillsdale, NJ.
[19] Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika82 711–732. · Zbl 0861.62023
[20] Jewell, C. P., Keeling, M. J. and Roberts, G. O. (2008). Predicting undetected infections during the 2007 foot and mouth disease outbreak. J. R. Soc. Interface6 1145–1151.
[21] Jewell, C. P., Kypraios, T., Neal, P. and Roberts, G. O. (2009). Bayesian analysis for emerging infectious diseases. Bayesian Anal.4 465–496. · Zbl 1330.62395
[22] Jombart, T., Didelot, X., Cauchemez, S., Viboud, F. C. and Ferguson, N. (2014). Bayesian reconstruction of disease outbreaks by combining epidemiologic and genomic data. PLoS Comput. Biol.10 e1003457.
[23] Keeling, M. J., Woolhouse, M. E. J., Shaw, D. J., Matthews, L., Chase-Topping, M., Haydon, D. T., Cornell, S. J., Kappey, J., Wilesmith, J. and Grenfell, B. T. (2001). Dynamics of the 2001 UK Foot and Mouth Epidemic: Stochastic dispersal in a heterogeneous landscape. Science294 813–817.
[24] King, A. A., Ionides, E. L., Pascual, M. and Bouma, M. J. (2008). Inapparent infections and cholera dynamics. Nature454 877–880.
[25] Knock, E. S. and O’Neill, P. D. (2014). Bayesian model choice for epidemic models with two levels of mixing. Biostatistics15 46–59.
[26] Lau, M. S. Y., Dalziel, B. D., Funk, S., McClelland, A., Tiffany, A., Riley, S., Metcalf, C., Jessica, E. and Grenfell, B. T. (2017). Spatial and temporal dynamics of superspreading events in the 2014–2015 West Africa Ebola epidemic. Proc. Natl. Acad. Sci. USA114 2337–2342.
[27] Lau, M. S. Y., Marion, G., Streftaris, G. and Gibson, G. J. (2014). New model diagnostics for spatio-temporal systems in ecology and epidemiology. J. R. Soc. Interface11 20131093. DOI:10.1098/rsif.2013.1093.
[28] Lau, M. S. Y., Marion, G., Streftaris, G. and Gibson, G. J. (2015). A systematic Bayesian integration of epidemiological and genetic data. PLoS Comput. Biol.11. e1004633. DOI:10.1371/journal.pcbi.1004633.
[29] McKinley, T., Cook, A. R. and Deardon, R. (2009). Inference in epidemic models without likelihoods. Int. J. Biostat.5 Art. 24, 39.
[30] Meng, X.-L. (1994). Posterior predictive \(p\)-values. Ann. Statist.22 1142–1160. · Zbl 0820.62027
[31] Meng, X.-L. and Vaida, F. (2006). Comment on article by Celeux et al. [MR2282197]. Bayesian Anal.1 687–698. · Zbl 1331.62338
[32] Monto, A. S., Koopman, J. S. and Longini, I. M. (1985). Tecumseh study of illness: XIII, influenza infection and disease, 1976–1981. Amer. J. Epidemiol.121 811–822.
[33] Morelli, M. J., Thébaud, G., Chadœuf, J., King, D. P., Haydon, D. T. and Soubeyrand, S. (2012). A Bayesian inference framework to reconstruct transmission trees using epidemiological and genetic data. PLoS Comput. Biol.8 e1002768, 14.
[34] Morton, A. and Finkenstädt, B. F. (2005). Discrete time modelling of disease incidence time series by using Markov chain Monte Carlo methods. J. R. Stat. Soc. Ser. C. Appl. Stat.54 575–594. · Zbl 05188699
[35] Neal, P. J. and Roberts, G. O. Statistical inference and model selection for the 1861 Hagelloch measles epidemic. Biostatistics5 249–261. · Zbl 1096.62123
[36] Neri, F. M., Cook, A. R., Gibson, G. J., Gottwald, T. R. and Gilligan, C. A. (2014). Bayesian analysis for inference of an emerging epidemic: Citrus canker in urban landscapes. PLoS Comput. Biol.10 e1003587. DOI:10.1371/journal.pcbi.1003587.
[37] O’Neill, P. D. and Becker, N. G. (2001). Inference for an epidemic when susceptibility varies. Biostatistics2 99–108. · Zbl 1017.62115
[38] O’Neill, P. D. and Roberts, G. O. (1999). Bayesian inference for partially observed epidemics. J. R. Stat. Soc., A162 121–129.
[39] Papaspiliopoulos, O., Roberts, G. O. and Sköld, M. (2007). A general framework for the parametrization of hierarchical models. Statist. Sci.22 59–73. · Zbl 1246.62195
[40] Parry, M., Gibson, G. J., Parnell, S., Gottwald, T. R., Irey, M. S., Gast, T. C. and Gilligan, C. A. (2014). Bayesian inference for an emerging arboreal epidemic in the presence of control. Proc. Natl. Acad. Sci. USA111 6258–6262.
[41] Sellke, T. (1983). On the asymptotic distribution of the size of a stochastic epidemic. J. Appl. Probab.20 390–394. · Zbl 0526.92024
[42] Spiegelhalter, D. J., Best, N. G., Carlin, B. P. and van der Linde, A. (2002). Bayesian measures of model complexity and fit. J. R. Stat. Soc. Ser. B. Stat. Methodol.64 583–639. · Zbl 1067.62010
[43] Streftaris, G. and Gibson, G. J. (2004). Bayesian analysis of experimental epidemics of foot-and-mouth disease. Proc. R. Soc. Lond., B Biol. Sci.271 1111–1117.
[44] Streftaris, G. and Gibson, G. J. (2012). Non-exponential tolerance to infection in epidemic systems—modelling, inference and assessment. Biostatistics13 580–593.
[45] Tanner, M. A. and Wong, W. H. (1987). The calculation of posterior distributions by data augmentation. J. Amer. Statist. Assoc.82 528–550. · Zbl 0619.62029
[46] Tildesley, M. J., Deardon, R., Savill, N. J., Bessell, P. R., Brooks, S. P., Woolhouse, M. E. J., Grenfell, B. T. and Keeling, M. J. (2008). Accuracy of models for the 2001 foot-and-mouth epidemic. Proc. R. Soc. Lond., B Biol. Sci.275 1459–1468.
[47] Verdinelli, I. and Wasserman, L. (1995). Computing Bayes factors using a generalization of the Savage–Dickey density ratio. J. Amer. Statist. Assoc.90 614–618. · Zbl 0826.62022
[48] Ypma, R., Bataille, A., Stegeman, A., Koch, G., Wallinga, J. and Van Ballegooijen, W. (2012). Unravelling transmission trees of infectious diseases by combining genetic and epidemiological data. Proc. R. Soc. Lond., B Biol. Sci.279 444–450.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.