×

An information-theoretic approach to assess practical identifiability of parametric dynamical systems. (English) Zbl 1343.62004

Summary: A new approach for assessing parameter identifiability of dynamical systems in a Bayesian setting is presented. The concept of Shannon entropy is employed to measure the inherent uncertainty in the parameters. The expected reduction in this uncertainty is seen as the amount of information one expects to gain about the parameters due to the availability of noisy measurements of the dynamical system. Such expected information gain is interpreted in terms of the variance of a hypothetical measurement device that can measure the parameters directly, and is related to practical identifiability of the parameters. If the individual parameters are unidentifiable, correlation between parameter combinations is assessed through conditional mutual information to determine which sets of parameters can be identified together. The information theoretic quantities of entropy and information are evaluated numerically through a combination of Monte Carlo and \(k\)-nearest neighbour methods in a non-parametric fashion. Unlike many methods to evaluate identifiability proposed in the literature, the proposed approach takes the measurement-noise into account and is not restricted to any particular noise-structure. Whilst computationally intensive for large dynamical systems, it is easily parallelisable and is non-intrusive as it does not necessitate re-writing of the numerical solvers of the dynamical system. The application of such an approach is presented for a variety of dynamical systems – ranging from systems governed by ordinary differential equations to partial differential equations – and, where possible, validated against results previously published in the literature.

MSC:

62B10 Statistical aspects of information-theoretic topics
62F15 Bayesian inference
37C10 Dynamics induced by flows and semiflows
60J70 Applications of Brownian motions and diffusion theory (population genetics, absorption problems, etc.)
94A17 Measures of information, entropy

Software:

npeet
PDFBibTeX XMLCite
Full Text: DOI HAL

References:

[1] Bellman, R.; Astrom, K. J., On structural identifiability, Math. Biosci., 7, 329-339 (1970)
[2] Cobelli, C.; Di Stefano, J. J., Parameter and structural identifiability concepts and ambiguities: a critical review and analysis, Am. J. Physiol. Regul. Integr. Comp. Physiol., 239, 1, R7-R24 (1980)
[3] Miao, H.; Xia, X.; Perelson, A. S.; Wu, H., On identifiability of nonlinear ODE models and applications in viral dynamics, SIAM Rev., 53, 1, 3-39 (2011) · Zbl 1215.34015
[4] Pohjanpalo, H., System identifiability based on power series expansion of solutions, Math. Biosci., 41, 21-33 (1978) · Zbl 0393.92008
[5] Nemcova, J., Structural identifiability of polynomial and rational systems, Math. Biosci., 223, 83-96 (2010) · Zbl 1183.92004
[6] Meshkat, N.; Eisemberg, M.; DiStefano, J. J., An algorithm for finding globally identifiable parameter combinations of nonlinear ode models using groebner bases, Math. Biosci., 222, 61-72 (2009) · Zbl 1179.93066
[7] Denis-Vidal, L.; Joly-Blanchard, G.; Noiret, C., Some effective approaches to check the identifiability of uncontrolled nonlinear systems, Math. Comput. Simulat., 57, 35-44 (2001) · Zbl 0984.93011
[8] Raue, A.; al., Structural and practical identifiability analysis of partly observed dynamical models by exploiting the profile likelihood., Bioinformatics, 25, 1923-1929 (2009)
[9] Eisenberg, M. C.; Hayashi, M. A.L., Determining identifiable parameter combinations using subset profiling, Math. Biosci., 256, 116-126 (2014) · Zbl 1330.92009
[10] Kaipio, J. P.; Somersalo, E., Statistical and Computational inverse problems (2004), Springer Verlag, New York
[11] Oden, T. J.; Prudencio, E. E.; Hawkins-Daarud, A., Selection and assessment of phenomenological models of tumor growth, Math. Model. Meth. Appl. Sci., 23, 07, 1309-1338 (2013) · Zbl 1301.92034
[12] Huan, X.; Marzouk, Y. M., Simulation-based optimal bayesian experimental design for nonlinear systems, J. Comput. Phys., 231, 288-317 (2013)
[13] Liepe, J.; Filippi, S.; Komorowski, M.; Stumpf, M. P., Maximizing the information content of experiments in system biology, Plos Comput. Biol., 9, 1, e1002888 (2013)
[14] Agapiou, S.; Larsson, S.; Stuart, A. M., Posterior consistency of the bayesian approach to linear ill-posed inverse problems, Stoch. Process. Appl., 123, 10, 3828-3860 (2013) · Zbl 1284.62289
[15] Iglesias, M. A.; Lin, K.; Stuart, A. M., Well-posed bayesian geometric inverse problems arising in subsurface flow, Inverse Prob., 30, 11, 114001 (2014) · Zbl 1304.35767
[16] Shannon, C. E., A mathematical theory of communication, Bell Syst. Tech. J., 27, 3, 379-423 (1948) · Zbl 1154.94303
[17] Jaynes, E. T., Information Theory and Statistical Mechanics (Notes by the lecturer), Statistical Physics 3 (1962 Brandeis Lectures), 181-218 (1963)
[18] Kullback, S.; Leibler, R. A., On information and sufficiency, Ann. Math. Stat., 22, 1, 79-86 (1951) · Zbl 0042.38403
[19] Kullback, S., Information theory and statistics (1997), Courier Dover Publications · Zbl 0149.37901
[20] Hobson, A., A new theorem of information theory, J. Stat. Phys., 1, 3, 383-391 (1969)
[21] Hobson, A.; Cheng, B.-K., A comparison of the shannon and kullback information measures, J. Stat. Phys., 7, 4, 301-310 (1973) · Zbl 1255.94042
[22] Cover, T. M.; Thomas, J. A., Elements of information theory (2012), John Wiley & Sons
[23] Kraskov, A.; Stögbauer, H.; Grassberger, P., Estimating mutual information, Phys. Rev. E, 69, 066138 (2004)
[24] Jensen, J. L.W. V., Sur les fonctions convexes et les ingalits entre les valeurs moyennes, Acta Math., 30, 1, 175-193 (1906) · JFM 37.0422.02
[25] Rinderknecht, S. L.; Albert, C.; Borsuk, M. E.; Schuwirth, N.; Künsch, H. R.; Reichert, P., The effect of ambiguous prior knowledge on bayesian model parameter inference and prediction, Environ. Model. Softw., 62, 0, 300-315 (2014)
[26] Ruggeri, F.; Insua, D. R.; Martn, J., Robust bayesian analysis, (Dey, D.; Rao, C., Bayesian Thinking Modeling and Computation. Bayesian Thinking Modeling and Computation, Handbook of Statistics, 25 (2005), Elsevier), 623-667
[27] Beirlant, J.; Dudewicz, E. J.; Györfi, L.; Van der Meulen, E. C., Nonparametric entropy estimation: An overview, Int. J. Math. Stat. Sci., 6, 1, 17-39 (1997) · Zbl 0882.62003
[28] Steuer, R.; Kurths, J.; Daub, C. O.; Weise, J.; Selbig, J., The mutual information: detecting and evaluating dependencies between variables, Bioinformatics, 18, suppl 2, S231-S240 (2002)
[29] Moon, Y.-I.; Rajagopalan, B.; Lall, U., Estimation of mutual information using kernel density estimators, Phys. Rev. E, 52, 3, 2318 (1995)
[30] Kozachenko, L. F.; Leonenko, N. N., Sample estimate of the entropy of a random vector, Probl. Inform. Transm., 23, 2, 9-16 (1987) · Zbl 0633.62005
[31] Singh, H.; Misra, N.; Hnizdo, V.; Fedorowicz, A.; Demchuk, E., Nearest neighbor estimates of entropy, Am. J. Math. Manag. Sci., 23, 3-4, 301-321 (2003)
[32] Westerhof, N.; Lankhaar, J.-W.; Westerhof, B., The arterial Windkessel, Med. Biol. Eng. Comput., 47, 2, 131-141 (2009)
[33] Vignon-Clementel, I. E.; Alberto Figueroa, C.; Jansen, K. E.; Taylor, C.a., Outflow boundary conditions for three-dimensional finite element modeling of blood flow and pressure in arteries, Comput. Meth. Appl. Mech. Eng., 195, 29-32, 3776-3796 (2006) · Zbl 1175.76098
[34] Thomaseth, K.; Cobelli, C., Generalized sensitivity functions in physiological system identification, Ann. Biomed. Eng., 27, 5, 607-616 (1999)
[35] Pant, S.; Fabrèges, B.; Gerbeau, J.-F.; Vignon-Clementel, I. E., A methodological paradigm for patient-specific multi-scale cfd simulations: from clinical measurements to parameter estimates for individual analysis, Int. J. Numer. Method. Biomed. Eng., 30, 12, 1614-1648 (2014)
[36] Greg Ver, S., Documention of the non-parametric entropy estimation toolbox (npeet), Online documentation (2014)
[37] Costa, J. A.; Hero, A. O., Geodesic entropic graphs for dimension and entropy estimation in manifold learning, IEEE Trans. Signal Process., 52, 8, 2210-2221 (2004) · Zbl 1369.68278
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.