×

Information theory, model error, and predictive skill of stochastic models for complex nonlinear systems. (English) Zbl 1260.62067

Summary: Many problems in complex dynamical systems involve metastable regimes despite nearly Gaussian statistics with underlying dynamics that is very different from the more familiar flows of molecular dynamics. There is significant theoretical and applied interest in developing systematic coarse-grained descriptions of the dynamics, as well as assessing their skill for both short- and long-range prediction. Clustering algorithms, combined with finite-state processes for the regime transitions, are a natural way to build such models objectively from data generated by either the true model or an imperfect model.
The main theme of this paper is the development of new practical criteria to assess the predictability of regimes and the predictive skill of such coarse-grained approximations through empirical information theory in stationary and periodically-forced environments. These criteria are tested on instructive idealized stochastic models utilizing \(K\)-means clustering in conjunction with running-average smoothing of the training and initial data for forecasts. A perspective on these clustering algorithms is explored here with independent interest, where improvement in the information content of finite-state partitions of phase space is a natural outcome of low-pass filtering through running averages. In applications with time-periodic equilibrium statistics, recently developed finite-element, bounded-variation algorithms for nonstationary autoregressive models are shown to substantially improve predictive skill beyond standard autoregressive models.

MSC:

62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62M20 Inference from stochastic processes and prediction
94A15 Information theory (general)
62B10 Statistical aspects of information-theoretic topics
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Lorenz, E. N., The predictability of a flow which possesses many scales of motion, Tellus, 21, 289-307 (1969)
[2] Epstein, E. S., Stochastic dynamic predictions, Tellus, 21, 739-759 (1969)
[3] Ruelle, D.; Takens, F., On the nature of turbulence, Comm. Math. Phys., 20, 167-192 (1971) · Zbl 0223.76041
[4] Vastano, J. A.; Swinney, H. L., Information transport in spatiotemporal systems, Phys. Rev. Lett., 60, 1773 (1988)
[5] Sobczyk, K., Information dynamics: premises, challenges and results, Mech. Syst. Signal Process., 15, 475-498 (2001)
[6] Majda, A. J.; Harlim, J., Information flow between subspaces of complex dynamical systems, Proc. Natl. Acad. Sci., 104, 9558-9563 (2007) · Zbl 1155.37048
[7] R. Kleeman, Information theory and dynamical system predictability, in: Isaac Newton Institute Preprint Series, NI10063, 2010, pp. 1-33.; R. Kleeman, Information theory and dynamical system predictability, in: Isaac Newton Institute Preprint Series, NI10063, 2010, pp. 1-33.
[8] Katsoulakis, M. A.; Majda, A. J.; Vlachos, D. G., Coarse-grained stochastic processes and Monte Carlo simulations in lattice systems, J. Comput. Phys., 186, 250-278 (2003) · Zbl 1034.82053
[9] Katsoulakis, M. A.; Plechác˘, P.; Sopasakis, A., Error analysis of coarse-graining for stochastic lattice dynamics, J. Numer. Anal., 44, 2270-2296 (2006) · Zbl 1140.82035
[10] Leung, L.-Y.; North, G. R., Information theory and climate prediction, J. Clim., 3, 5-14 (1990)
[11] Schneider, T.; Griffies, S. M., A conceptual framework for predictability studies, J. Clim., 12, 3133-3155 (1999)
[12] Kleeman, R., Measuring dynamical prediction utility using relative entropy, J. Atmos. Sci., 59, 2057-2072 (2002)
[13] Majda, A. J.; Kleeman, R.; Cai, D., A mathematical framework for predictability through relative entropy, Methods Appl. Anal., 9, 425-444 (2002) · Zbl 1084.94010
[14] Roulston, M.; Smith, L., Evaluating probabilistic forecasts using information theory, Mon. Weather Rev., 130, 1653-1660 (2002)
[15] DelSole, T., Predictability and information theory, part I: measures of predictability, J. Atmos. Sci., 61, 2425-2440 (2004)
[16] DelSole, T., Predictability and information theory, part II: imperfect models, J. Atmos. Sci., 62, 3368-3381 (2005)
[17] Young, R. M.B.; Read, P. L., Breeding and predictability in the baroclinic rotating annulus using a perfect model, Nonlinear Processes Geophys., 15, 469-487 (2008) · Zbl 1152.76495
[18] Majda, A. J.; Gershgorin, B., Quantifying uncertainty in climate change science through empirical information theory, Proc. Natl. Acad. Sci., 107, 14958-14963 (2010)
[19] Teng, H.; Branstator, G., Initial-value predictability of prominent modes of North Pacific subsurface temperature in a CGCM, Climate Dyn., 36, 1813-1834 (2011)
[20] Majda, A. J.; Gershgorin, B., Improving model fidelity and sensitivity for complex systems through empirical information theory, Proc. Natl. Acad. Sci., 108, 10044-10049 (2011) · Zbl 1256.94026
[21] Deuflhard, P.; Dellnitz, M.; Junge, O.; Schütte, C., Computation of essential molecular dynamics by subdivision techniques I: basic concept, Lect. Notes Comp. Sci. Eng., 4, 98 (1999) · Zbl 0966.81067
[22] Deuflhard, P.; Huisinga, W.; Fischer, A.; Schütte, C., Identification of almost invariant aggregates in reversible nearly uncoupled Markov chains, Linear Algebra Appl., 315, 39 (2000) · Zbl 0963.65008
[23] Horenko, I.; Schütte, C., On metastable conformation analysis of nonequilibrium biomolecular time series, Multiscale Model. Simul., 8, 701-716 (2010) · Zbl 1284.62048
[24] Tsay, R. S., Analysis of Financial Time Series (2010), Wiley: Wiley Hoboken · Zbl 1209.91004
[25] Crommelin, D. T.; Vanden-Eijnden, E., Fitting timeseries by continuous-time Markov chains: a quadratic programming approach, J. Comput. Phys., 217, 782-805 (2006) · Zbl 1102.65009
[26] Metzner, P.; Dittmer, E.; Jahnke, T.; Schütte, C., Generator estimation of Markov jump processes based on incomplete observations equidistant in time, J. Comput. Phys., 227, 353-375 (2007) · Zbl 1128.65012
[27] Horenko, I., On simultaneous data-based dimension reduction and hidden phase identification, J. Atmos. Sci., 65, 1941-1954 (2008)
[28] Bröcker, J.; Engster, D.; Parlitz, U., Probabilistic evaluation of time series models: a comparison of several approaches, Chaos, 19, 04130 (2009)
[29] Horenko, I., Finite element approach to clustering of multidimensional time series, SIAM J. Sci. Comput., 32, 62-83 (2010) · Zbl 1206.62150
[30] J. de Wiljes, A.J. Majda, I. Horenko, An adaptive Markov chain Monte Carlo approach to time series clustering with regime transition behavior, SIAM J. Multiscale Model. Simul. (2010) (submitted for publication).; J. de Wiljes, A.J. Majda, I. Horenko, An adaptive Markov chain Monte Carlo approach to time series clustering with regime transition behavior, SIAM J. Multiscale Model. Simul. (2010) (submitted for publication). · Zbl 06297626
[31] Horenko, I., Parameter identification in nonstationary Markov chains with external impact and its application to computational sociology, SIAM J. Multiscale Model. Simul., 9, 1700-1726 (2011) · Zbl 1244.60070
[32] Berner, J.; Branstator, G., Linear and nonlinear signatures in planetary wave dynamics of an AGCM: probability density functions, J. Atmos. Sci., 64, 117-136 (2007)
[33] Majda, A. J.; Franzke, C.; Fischer, A.; Crommelin, D. T., Distinct metastable atmospheric regimes despite nearly Gaussian statistics: a paradigm model, Proc. Natl. Acad. Sci., 103, 8309-8314 (2006) · Zbl 1160.86304
[34] Franzke, C.; Majda, A. J.; Branstator, G., The origin of nonlinear signatures of planetary wave dynamics: mean phase space tendencies and contributions from non-Gaussianity, J. Atmos. Sci., 64, 3988 (2007)
[35] Majda, A. J.; Wang, X., Linear response theory for statistical ensembles in complex systems with time-periodic forcing, Commun. Math. Sci., 8, 145-172 (2010) · Zbl 1201.37109
[36] Horenko, I., On the identification of nonstationary factor models and their application to atmospheric data analysis, J. Atmos. Sci., 67, 1559-1574 (2010)
[37] Franzke, C.; Crommelin, D.; Fischer, A.; Majda, A. J., A hidden Markov model perspective on regimes and metastability in atmospheric flows, J. Clim., 21, 1740-1757 (2008)
[38] Penland, C., Random forcing and forecasting using principal oscillation pattern analysis, Mon. Weather Rev., 117, 2165-2185 (1989)
[39] Majda, A. J.; Timofeyev, I. I.; Vanden Eijnden, E., Systematic strategies for stochastic mode reduction in climate, J. Atmos. Sci., 60, 1705 (2003)
[40] Franzke, C.; Horenko, I.; Majda, A. J.; Klein, R., Systematic metastable regime identification in an AGCM, J. Atmos. Sci., 66, 1997-2012 (2009)
[41] Horenko, I., On robust estimation of low-frequency variability trends in discrete Markovian sequences of atmospheric circulation patterns, J. Atmos. Sci., 66, 2059-2072 (2009)
[42] Horenko, I., On clustering of non-stationary meteorological time series, Dyn. Atmos. Oceans, 49, 164-187 (2010)
[43] Abramov, R. V.; Majda, A. J.; Kleeman, R., Information theory and predictability for low-frequency variability, J. Atmos. Sci., 62, 65-87 (2005)
[44] DelSole, T.; Tippett, M. K., Predictability: recent insights from information theory, Rev. Geophys., 45, RG4002 (2007)
[45] DelSole, T.; Shukla, J., Model fidelity versus skill in seasonal forecasting, J. Clim., 23, 4794-4806 (2010)
[46] Giannakis, D.; Majda, A. J., Quantifying the predictive skill in long-range forecasting, part I: coarse-grained predictions in a simple ocean model, J. Clim., 25, 1793-1813 (2011)
[47] Giannakis, D.; Majda, A. J., Quantifying the predictive skill in long-range forecasting, part II: model error in coarse-grained Markov models with application to ocean-circulation regimes, J. Clim., 25, 1814-1826 (2011)
[48] Majda, A. J.; Gershgorin, B.; Yuan, Y., Low-frequency climate response and fluctuation-dissipation theorems: theory and practice, J. Atmos. Sci., 67, 1186 (2010)
[49] Akaike, H., Information theory and an extension of the maximum likelihood principle, (Petrov, B. N.; Caski, F., Proceedings of the Second International Symposium on Information Theory (1973), Akademiai Kiado: Akademiai Kiado Budapest), 610 · Zbl 0283.62006
[50] McQuarrie, A. D.R.; Tsai, C.-L., Regression and Time Series Model Selection (1998), World Scientific: World Scientific Singapore · Zbl 0907.62095
[51] Cover, T. A.; Thomas, J. A., Elements of Information Theory (2006), Wiley-Interscience: Wiley-Interscience Hoboken · Zbl 1140.94001
[52] Horenko, I., Nonstationarity in multifactor models of discrete jump processes, memory and application to cloud modeling, J. Atmos. Sci. (2011), Early online release
[53] Gneiting, T.; Raftery, A. E., Strictly proper scoring rules, prediction and estimation, J. Amer. Statist. Assoc., 102, 359-378 (2007) · Zbl 1284.62093
[54] Bröcker, J., Reliability, sufficiency and the decomposition of proper scores, Q. J. R. Meteorol. Soc., 135, 1512-1519 (2009)
[55] D. Madigan, A. Raftery, C. Volinsky, J. Hoeting, Bayesian model averaging, in: Proceedings of the AAAI Workshop on Integrating Multiple Learned Models, Portland, OR, pp. 77-83.; D. Madigan, A. Raftery, C. Volinsky, J. Hoeting, Bayesian model averaging, in: Proceedings of the AAAI Workshop on Integrating Multiple Learned Models, Portland, OR, pp. 77-83. · Zbl 1059.62525
[56] Hoeting, J. A.; Madigan, D.; Raftery, A. E.; Volinsky, C. T., Bayesian model averaging: a tutorial, Stat. Sci., 14, 382-401 (1999) · Zbl 1059.62525
[57] Raftery, A. E.; Gneiting, T.; Balabdaoui, F.; Polakowski, M., Using Bayesian model averaging to calibrate forecast ensembles, Mon. Weather Rev., 133, 1155-1173 (2005)
[58] Branstator, G.; Berner, J., Linear and nonlinear signatures in the planetary wave dynamics of an AGCM: phase space tendencies, J. Atmos. Sci., 62, 1792-1811 (2005)
[59] Majda, A. J.; Franzke, C.; Crommelin, D., Normal forms for reduced stochastic climate models, Proc. Natl. Acad. Sci., 106, 3649 (2009) · Zbl 1202.86011
[60] DeGroot, M. H.; Fienberg, S. E., Assessing probability assessors: calibration and refinements, (Gupta, S. S.; Berger, J. O., Statistical Decision Theory and Related Topics III, Vol. 1 (1982), Academic Press: Academic Press New York), 291-314 · Zbl 0581.62003
[61] Joe, H., Relative entropy measures of multivariate dependence, J. Amer. Stat. Assoc., 84, 157-164 (1989) · Zbl 0677.62054
[62] Silverman, B. W., (Density Estimation for Statistics and Data Analysis. Density Estimation for Statistics and Data Analysis, Monographs on Statistics and Applied Probability, vol. 26 (1986), Chapman & Hall/CRC: Chapman & Hall/CRC Boca Raton) · Zbl 0617.62042
[63] Duda, R. O.; Hart, P. E.; Stork, D. G., Pattern Classification (2000), Wiley-Interscience: Wiley-Interscience New York
[64] Khan, S.; Bandyopadhyay, S.; Ganguly, A. R.; Saigal, S.; Erickson III, D. J.; Protopopescu, V.; Ostrouchov, G., Relative performance of mutual information estimation methods for quantifying the dependence among short and noisy data, Phys. Rev. E, 76, 026209 (2007)
[65] Majda, A. J.; Abramov, R.; Gershgorin, B., High skill in low-frequency climate response through fluctuation dissipation theorems despite structural instability, Proc. Natl. Acad. Sci., 107, 581-586 (2010) · Zbl 1205.86025
[66] P. Metzner, L. Putzig, I. Horenko, Analysis of persistent non-stationary time series and applications, Commun. Appl. Math. Comput. Sci. (2012) (in press).; P. Metzner, L. Putzig, I. Horenko, Analysis of persistent non-stationary time series and applications, Commun. Appl. Math. Comput. Sci. (2012) (in press). · Zbl 1275.62067
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.