×

Prediction uncertainty and optimal experimental design for learning dynamical systems. (English) Zbl 1374.37123

Summary: Dynamical systems are frequently used to model biological systems. When these models are fit to data, it is necessary to ascertain the uncertainty in the model fit. Here, we present prediction deviation, a metric of uncertainty that determines the extent to which observed data have constrained the model’s predictions. This is accomplished by solving an optimization problem that searches for a pair of models that each provides a good fit for the observed data, yet has maximally different predictions. We develop a method for estimating a priori the impact that additional experiments would have on the prediction deviation, allowing the experimenter to design a set of experiments that would most reduce uncertainty. We use prediction deviation to assess uncertainty in a model of interferon-alpha inhibition of viral infection, and to select a sequence of experiments that reduces this uncertainty. Finally, we prove a theoretical result which shows that prediction deviation provides bounds on the trajectories of the underlying true model. These results show that prediction deviation is a meaningful metric of uncertainty that can be used for optimal experimental design.{
©2016 American Institute of Physics}

MSC:

37N20 Dynamical systems in other branches of physics (quantum mechanics, general relativity, laser physics)
92C42 Systems biology, networks
PDFBibTeX XMLCite
Full Text: DOI arXiv Link

References:

[1] Brackley, C. A.; Ebenhh, O.; Grebogi, C.; Kurths, J.; de Moura, A.; Romano, M. C.; Thiel, M., Introduction to focus issue: Dynamics in systems biology, Chaos, 20, 045101 (2010) · doi:10.1063/1.3530126
[2] Raue, A.; Kreutz, C.; Maiwald, T.; Bachmann, J.; Schilling, M.; Klingmüller, U.; Timmer, J., Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood, Bioinformatics, 25, 1923-1929 (2009) · doi:10.1093/bioinformatics/btp358
[3] Bellu, G.; Saccomani, M. P.; Audoly, S.; D’Angiò, L., DAISY: A new software tool to test global identifiability of biological and physiological systems, Comput. Methods Programs Biomed., 88, 52-61 (2007) · doi:10.1016/j.cmpb.2007.07.002
[4] Chiş, O.; Banga, J. R.; Balsa-Canto, E., GenSSI: A software toolbox for structural identifiability analysis of biological models, Bioinformatics, 27, 2610-2611 (2011) · doi:10.1093/bioinformatics/btr431
[5] Sedoglavic, A., A probabilistic algorithm to test local algebraic observability in polynomial time (2001) · Zbl 1356.93017
[6] Gutenkunst, R. N.; Waterfall, J. J.; Casey, F. P.; Brown, K. S.; Myers, C. R.; Sethna, J. P., Universally sloppy parameter sensitivities in systems biology models, PLoS Comput. Biol., 3, e189 (2007) · doi:10.1371/journal.pcbi.0030189
[7] Efron, B.; Tibshirani, R., Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy, Stat. Sci., 1, 54-77 (1986) · Zbl 0955.62560 · doi:10.1214/ss/1177013815
[8] See supplementary material at for additional figures showing prediction deviation models and estimated experiment impact models for the Lorenz system.
[9] Browne, E. P.; Letham, B.; Rudin, C., A computational model of inhibition of HIV-1 by interferon-alpha, PLoS One, 11, e0152316 (2016) · doi:10.1371/journal.pone.0152316
[10] Kreutz, C.; Raue, A.; Timmer, J., Likelihood based observability analysis and confidence intervals for predictions of dynamic models, BMC Syst. Biol., 6, 120 (2012) · doi:10.1186/1752-0509-6-120
[11] St John, P. C.; Doyle, F. J., Estimating confidence intervals in predicted responses for oscillatory biological models, BMC Syst. Biol., 7, 71 (2013) · doi:10.1186/1752-0509-7-71
[12] Vanlier, J.; Tiemann, C. A.; Hilbers, P. A. J.; van Riel, N. A. W., A Bayesian approach to targeted experiment design, Bioinformatics, 28, 1136-1142 (2012) · doi:10.1093/bioinformatics/bts092
[13] Vanlier, J.; Tiemann, C. A.; Hilbers, P. A. J.; van Riel, N. A. W., Parameter uncertainty in biochemical models described by ordinary differential equations, Math. Biosci., 246, 305-314 (2013) · Zbl 1283.92046 · doi:10.1016/j.mbs.2013.03.006
[14] Bandara, S.; Schlöder, J. P.; Eils, R.; Bock, H. G.; Meyer, T., Optimal experimental design for parameter estimation of a cell signaling model, PLoS Comput. Biol., 5, e1000558 (2009) · doi:10.1371/journal.pcbi.1000558
[15] Raue, A.; Becker, V.; Klingmüller, U.; Timmer, J., Identifiability and observability analysis for experimental design in nonlinear dynamical models, Chaos, 20, 045105 (2010) · Zbl 1311.92066 · doi:10.1063/1.3528102
[16] Transtrum, M. K.; Qiu, P., Optimal experiment selection for parameter estimation in biological differential equation models, BMC Bioinf., 13, 181 (2012) · doi:10.1186/1471-2105-13-181
[17] Flassig, R. J.; Sundmacher, K., Optimal design of stimulus experiments for robust discrimination of biochemical reaction networks, Bioinformatics, 28, 3089-3096 (2012) · doi:10.1093/bioinformatics/bts585
[18] Vanlier, J.; Tiemann, C. A.; Hilbers, P. A. J.; van Riel, N. A. W., Optimal experiment design for model selection in biochemical networks, BMC Syst. Biol., 8, 20 (2014) · doi:10.1186/1752-0509-8-20
[19] Busetto, A. G.; Hauser, A.; Krummenacher, G.; Sunnåker, M.; Dimopoulos, S.; Ong, C. S.; Stelling, J.; Buhmann, J. M., Near-optimal experimental design for model selection in systems biology, Bioinformatics, 29, 2625-2632 (2013) · doi:10.1093/bioinformatics/btt436
[20] Daunizeau, J.; Preuschoff, K.; Friston, K.; Stephan, K., Optimizing experimental design for comparing models of brain function, PLoS Comput. Biol., 7, e1002280 (2011) · doi:10.1371/journal.pcbi.1002280
[21] Skanda, D.; Lebiedz, D., An optimal experimental design approach to model discrimination in dynamic biochemical systems, Bioinformatics, 26, 939-945 (2010) · doi:10.1093/bioinformatics/btq074
[22] Kreutz, C.; Timmer, J., Systems biology: experimental design, FEBS J., 276, 923-942 (2009) · doi:10.1111/j.1742-4658.2008.06843.x
[23] Casey, F. P.; Baird, D.; Feng, Q.; Gutenkunst, R. N.; Waterfall, J. J.; Myers, C. R.; Brown, K. S.; Cerione, R. A.; Sethna, J. P., Optimal experimental design in an epidermal growth factor receptor signalling and down-regulation model, IET Syst. B., 1, 190-202 (2007) · doi:10.1049/iet-syb:20060065
[24] Myers, C. R.; Gutenkunst, R. N.; Sethna, J. P., Python unleashed on systems biology, Comput. Sci. Eng., 9, 34-37 (2007) · doi:10.1109/MCSE.2007.60
[25] Gutenkunst, R. N.; Atlas, J. C.; Casey, F. P.; Daniels, B. C.; Kuczenski, R. S.; Waterfall, J. J.; Myers, C. R.; Sethna, J. P., Sloppycell (2007)
[26] Nash, S. G., Newton-type minimization via the Lanczos method, SIAM J. Numer. Anal., 21, 770-778 (1984) · Zbl 0558.65041 · doi:10.1137/0721052
[27] Nocedal, J.; Wright, S., Numerical Optimization (2006) · Zbl 1104.65059
[28] Shaked, M.; Shanthikumar, J. G., Stochastic Orders (2007)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.