zbMATH — the first resource for mathematics

Separable nonlinear least-squares parameter estimation for complex dynamic systems. (English) Zbl 1435.92003
Summary: Nonlinear dynamic models are widely used for characterizing processes that govern complex biological pathway systems. Over the past decade, validation and further development of these models became possible due to data collected via high-throughput experiments using methods from molecular biology. While these data are very beneficial, they are typically incomplete and noisy, which renders the inference of parameter values for complex dynamic models challenging. Fortunately, many biological systems have embedded linear mathematical features, which may be exploited, thereby improving fits and leading to better convergence of optimization algorithms. In this paper, we explore options of inference for dynamic models using a novel method of separable nonlinear least-squares optimization and compare its performance to the traditional nonlinear least-squares method. The numerical results from extensive simulations suggest that the proposed approach is at least as accurate as the traditional nonlinear least-squares, but usually superior, while also enjoying a substantial reduction in computational time.
92B15 General biostatistics
62P10 Applications of statistics to biology and medical sciences; meta analysis
92C42 Systems biology, networks
62F10 Point estimation
Full Text: DOI
[1] Lotka, A. J., Elements of Mathematical Biology (1956), New York, NY, USA: Dover Publications Inc., New York, NY, USA · Zbl 0074.14404
[2] May, R. M., Stability and Complexity in Model Ecosystems. With a New Introduction by the Author (2001), Princeton, NJ, USA: Princeton University Press, Princeton, NJ, USA
[3] Peschel, M.; Mende, W., The Predator-Prey Model: Do We Live in a Volterra World? (1986), New York, NY, USA: Springer, New York, NY, USA · Zbl 0576.92001
[4] Volterra, V., Fluctuations in the abundance of a species considered mathematically, Nature, 118, 2972, 558-560 (1926) · JFM 52.0453.03
[5] Savageau, M. A., Biochemical systems analysis, A Study of Function and Design in Molecular Biology, Vol. 6739 of Advanced Book Program (1976), Boston, MA, USA: Addison-Wesley Publishing Company, Boston, MA, USA
[6] Voit, E. O., Biochemical systems theory: a review, ISRN Biomathematics, 2013 (2013) · Zbl 1268.92045
[7] Chou, I.-C.; Voit, E. O., Recent developments in parameter estimation and structure identification of biochemical and genomic systems, Mathematical Biosciences, 219, 2, 57-83 (2009) · Zbl 1168.92019
[8] Gennemark, P.; Wedelin, D., Efficient algorithms for ordinary differential equation model identification of biological systems, IET Systems Biology, 1, 2, 120-129 (2007)
[9] Girolami, M.; Calderhead, B., Riemann manifold Langevin and Hamiltonian Monte Carlo methods, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73, 2, 123-214 (2011) · Zbl 1411.62071
[10] McGoff, K.; Mukherjee, S.; Pillai, N., Statistical inference for dynamical systems: a review, Statistics Surveys, 9, 209-252 (2015) · Zbl 1327.62458
[11] Ramsay, J.; Hooker, G., Dynamic Data Analysis. Modeling Data with Differential Equations (2017), New York, NY, USA: Springer, New York, NY, USA
[12] Schittkowski, K., Numerical Data Fitting in Dynamical Systems. A Practical Introduction with Applications and Software (2002), Dordrecht, Netherlands: Kluwer Academic Publishers, Dordrecht, Netherlands
[13] Voit, E. O.; Almeida, J., Decoupling dynamical systems for pathway identification from metabolic profiles, Bioinformatics, 20, 11, 1670-1681 (2004)
[14] Dolatshahi, S.; Voit, E. O., Identification of metabolic pathway systems, Frontiers in Genetics, 7, 6 (2016)
[15] Faraji, M.; Voit, E. O., Stepwise inference of likely dynamic flux distributions from metabolic time series data, Bioinformatics, 33, 14, 2165-2172 (2017)
[16] Goel, G.; Chou, I.-C.; Voit, E. O., System estimation from metabolic time-series data, Bioinformatics, 24, 21, 2505-2511 (2008)
[17] Varah, J. M., A spline least squares method for numerical parameter estimation in differential equations, SIAM Journal on Scientific and Statistical Computing, 3, 1, 28-46 (1982) · Zbl 0481.65050
[18] Voit, E. O., Computational Analysis of Biochemical Systems: A Practical Guide for Biochemists and Molecular Biologists (2000), Cambridge, UK: Cambridge University Press, Cambridge, UK
[19] Himmelblau, D. M.; Jones, C. R.; Bischoff, K. B., Determination of rate constants for complex kinetics models, Industrial & Engineering Chemistry Fundamentals, 6, 4, 539-543 (1967)
[20] Dattner, I.; Klaassen, C. A. J., Optimal rate of direct estimators in systems of ordinary differential equations linear in functions of the parameters, Electronic Journal of Statistics, 9, 2, 1939-1973 (2015) · Zbl 1327.62120
[21] Golub, G. H.; Pereyra, V., The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate, SIAM Journal on Numerical Analysis, 10, 2, 413-432 (1973) · Zbl 0258.65045
[22] Ruhe, A.; Wedin, P. Å., Algorithms for separable nonlinear least squares problems, SIAM Review, 22, 3, 318-337 (1980) · Zbl 0466.65039
[23] Dattner, I.; Miller, E.; Petrenko, M.; Kadouri, D. E.; Jurkevitch, E.; Huppert, A., Modelling and parameter inference of predator-prey dynamics in heterogeneous environments using the direct integral approach, Journal of The Royal Society Interface, 14, 126 (2017)
[24] Wu, L.; Qiu, X.; Yuan, Y.-X.; Wu, H., Parameter estimation and variable selection for big systems of linear ordinary differential equations: a matrix-based approach, Journal of the American Statistical Association, 114, 526, 657-667 (2019) · Zbl 1420.62102
[25] Yaari, R.; Dattner, I., Simode: statistical inference for systems of ordinary differential equations using separable integral-matching, R package version 1.1.4 (January 2019), https://CRAN.R-project.org/package=simode
[26] Golub, G.; Pereyra, V., Separable nonlinear least squares: the variable projection method and its applications, Inverse Problems, 19, 2, 1-26 (2003) · Zbl 1022.65014
[27] Mullen, K. M., Separable nonlinear models: theory, implementation and applications in physics and chemistry (2008), Amsterdam, Netherlands: Vrije Universiteit, Amsterdam, Netherlands, Ph.D. thesis
[28] Chung, J.; Nagy, J. G., An efficient iterative approach for large-scale separable nonlinear inverse problems, SIAM Journal on Scientific Computing, 31, 6, 4654-4674 (2010) · Zbl 1205.65160
[29] Gan, M.; Chen, C. L. P.; Chen, G.-Y.; Chen, L., On some separated algorithms for separable nonlinear least squares problems, IEEE Transactions on Cybernetics, 48, 10, 2866-2874 (2018)
[30] Erichson, N. B.; Zheng, P.; Manohar, K.; Brunton, S. L.; Kutz, J. N.; Aravkin, A. Y., Sparse principal component analysis via variable projection (2018), https://arxiv.org/abs/1804.00341
[31] Basu, S.; Bresler, Y., The stability of nonlinear least squares problems and the Cramér-Rao bound, IEEE Transactions on Signal Processing, 48, 12, 3426-3436 (2000) · Zbl 1002.94004
[32] Hirsch, M. W.; Devaney, R. L.; Smale, S., Differential Equations, Dynamical Systems, and Linear Algebra, 60 (1974), Cambridge, MA, USA: Academic Press, Cambridge, MA, USA
[33] Fan, J.; Gijbels, I., Local Polynomial Modelling and its Applications (1996), London, UK: Chapman & Hall, London, UK, vol. 66 of Monographs on Statistics and Applied Probability · Zbl 0873.62037
[34] Green, P. J.; Silverman, B. W., Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach (1994), London, UK: Chapman & Hall, London, UK, vol. 58 of Monographs on Statistics and Applied Probability · Zbl 0832.62032
[35] Wasserman, L., All of Nonparametric Statistics (2006), New York, NY, USA: Springer, New York, NY, USA · Zbl 1099.62029
[36] Brunel, N. J.-B., Parameter estimation of ODEs via nonparametric estimators, Electronic Journal of Statistics, 2, 1242-1267 (2008) · Zbl 1320.62063
[37] Gugushvili, S.; Klaassen, C. A. J., \( \sqrt{n} \)-consistent parameter estimation for systems of ordinary differential equations: bypassing numerical integration via smoothing, Bernoulli, 18, 3, 1061-1098 (2012) · Zbl 1257.49033
[38] Vilela, M.; Borges, C. C. H.; Vinga, S., Automated smoother for the numerical decoupling of dynamics models, BMC Bioinformatics, 8, 1, 305 (2007)
[39] Chen, S.; Shojaie, A.; Witten, D. M., Network reconstruction from high-dimensional ordinary differential equations, Journal of the American Statistical Association, 112, 520, 1697-1707 (2017)
[40] Dattner, I., A model-based initial guess for estimating parameters in systems of ordinary differential equations, Biometrics, 71, 4, 1176-1184 (2015) · Zbl 1419.62083
[41] Dattner, I.; Gugushvili, S., Application of one-step method to parameter estimation in ODE models, Statistica Neerlandica, 72, 2, 126-156 (2018)
[42] Dattner, I.; Huppert, A., Modern statistical tools for inference and prediction of infectious diseases using mathematical models, Statistical Methods in Medical Research, 27, 7, 1927-1929 (2018)
[43] Vissing Mikkelsen, F.; Hansen, N. R., Learning large scale ordinary differential equation systems (2017), https://arxiv.org/abs/1710.09308
[44] Vujačić, I.; Dattner, I.; González, J.; Wit, E., Time-course window estimator for ordinary differential equations linear in the parameters, Statistics and Computing, 25, 6, 1057-1070 (2015) · Zbl 1331.62109
[45] Yaari, R.; Dattner, I., Simode: R package for statistical inference of ordinary differential equations using separable integral-matching (2018), https://joss.theoj.org/papers/10.21105/joss.01850
[46] Yaari, R.; Dattner, I.; Huppert, A., A two-stage approach for estimating the parameters of an age-group epidemic model from incidence data, Statistical Methods in Medical Research, 27, 7, 1999-2014 (2018)
[47] Lawton, W. H.; Sylvestre, E. A., Elimination of linear parameters in nonlinear regression, Technometrics, 13, 3, 461-467 (1971) · Zbl 0219.62014
[48] Wickham, H., Ggplot2. Elegant Graphics for Data Analysis (2009), New York, NY, USA: Springer, New York, NY, USA · Zbl 1170.62004
[49] Johnstone, I. M.; Silverman, B. W., Empirical Bayes selection of wavelet thresholds, The Annals of Statistics, 33, 4, 1700-1752 (2005) · Zbl 1078.62005
[50] Hethcote, H. W., The mathematics of infectious diseases, SIAM Review, 42, 4, 599-653 (2000) · Zbl 0993.92033
[51] FitzHugh, R., Impulses and physiological states in theoretical models of nerve membrane, Biophysical Journal, 1, 6, 445-466 (1961)
[52] FitzHugh, R.; Schwan, H. P., Mathematical models of excitation and propagation in nerve, Biological Engineering, 1-85 (1969), New York, NY, USA: McGraw-Hill, New York, NY, USA, vol. 9 of Inter-university electronics series
[53] Nagumo, J.; Arimoto, S.; Yoshizawa, S., An active pulse transmission line simulating nerve axon, Proceedings of the IRE, 50, 10, 2061-2070 (1962)
[54] Hodgkin, A. L.; Huxley, A. F., A quantitative description of membrane current and its application to conduction and excitation in nerve, The Journal of Physiology, 117, 4, 500-544 (1952)
[55] Ramsay, J. O.; Hooker, G.; Campbell, D.; Cao, J., Parameter estimation for differential equations: a generalized smoothing approach, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69, 5, 741-796 (2007)
[56] Campbell, D.; Steele, R. J., Smooth functional tempering for nonlinear differential equation models, Statistics and Computing, 22, 2, 429-443 (2012) · Zbl 1322.62011
[57] Tufte, E. R., The Visual Display of Quantitative Information (2001), Cheshire, CT, USA: Graphics Press, Cheshire, CT, USA
[58] Venables, W. N.; Ripley, B. D., Modern Applied Statistics with S. Statistics and Computing (2002), New York, NY, USA: Springer, New York, NY, USA · Zbl 1006.62003
[59] Mullen, K.; van Stokkum, I., TIMP: an R package for modeling multi-way spectroscopic measurements, Journal of Statistical Software, 18, 3, 1-46 (2007)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.