Regularized local linear prediction of chaotic time series. (English) Zbl 0931.37037

Summary: Local linear prediction, based on the ordinary least squares (OLS) approach, is one of several methods that have been applied to prediction of chaotic time series. Apart from potential linearization errors, a drawback of this approach is the high variance of the predictions under certain conditions. Here, a different set of so-called linear regularization techniques, originally derived to solve ill-posed regression problems, are compared to OLS for chaotic time series corrupted by additive measurement noise. These methods reduce the variance compared to OLS, but introduce more bias. A main tool of analysis is the singular value decomposition (SVD), and a key to successful regularization is to damp the higher order SVD components. Several of the methods achieve improved prediction compared to OLS for synthetic noise-corrupted data from well-known chaotic systems. Similar results were found for real-world data from the R-R intervals of ECG signals. Good results are also obtained for real sunspot data, compared to published predictions using nonlinear techniques.


37M10 Time series analysis of dynamical systems
62M20 Inference from stochastic processes and prediction
93C99 Model systems in control theory
37D45 Strange attractors, chaotic dynamics of systems with hyperbolic behavior
Full Text: DOI


[1] Weigend, A. S.; Gershenfeld, N. A., Time Series Prediction: Forecasting the Future and Understanding the Past (1994), Addison-Wesley: Addison-Wesley Reading, MA
[2] Tsonis, A. A., Chaos: From Theory to Applications (1992), Plenum Press: Plenum Press New York
[3] Lillekjendlie, B.; Kugiumtzis, D.; Christophersen, N., Chaotic time series Part II: System identification and prediction, Modeling, Identification and Control, 15, 4, 225-243 (1994) · Zbl 0850.93869
[4] Farmer, J. D.; Sidorowich, J. J., Predicting chaotic time series, Phys. Rev. Lett., 59, 845-848 (1987)
[5] (Brown, P. J., Measurement, Regression, and Calibration (1993), Oxford University Press: Oxford University Press Oxford) · Zbl 0829.62064
[6] Golub, G. H.; Van Loan, C. F., An analysis of the total least squares problem, SIAM J. Numer. Anal., 17, 6, 883-893 (1980) · Zbl 0468.65011
[7] Sauer, T., Time series prediction by using delay coordinate embedding, (Weigend, A. S.; Gershenfeld, N. A., Time Series Prediction: Forecasting the Future and Understanding the Past (1994), Addison-Wesley: Addison-Wesley Reading, MA), 175-193
[8] Kostelich, E. J.; Schreiber, T., Noise reduction in chaotic time-series data: A survey of common methods, Phys. Rev. E, 48, 3, 1752-1763 (1993)
[9] Lingjærde, O. C.; Christophersen, N., Shrinkage Properties of Partial Least Squares (1997), manuscript
[10] Packard, N. H.; Crutchfield, J. P.; Farmer, J. D.; Shaw, R. S., Geometry from a time series, Phys. Rev. Lett., 45, 712 (1980)
[11] Takens, F., Detecting strange attractors in turbulence, (Rand, D. A.; Young, L.-S., Dynamical Systems and Turbulence (Warwick, 1980). Dynamical Systems and Turbulence (Warwick, 1980), Lecture Notes in Mathematics, Vol. 898 (1981), Springer: Springer Berlin), 366-381
[12] (Fuller, W. A., Measurement Error Models (1987), Wiley: Wiley New York) · Zbl 0800.62413
[13] Scharf, L. L., Statistical Signal Processing: Detection, Estimation and Time Series Analysis (1991), Addison-Wesley: Addison-Wesley Reading, MA · Zbl 1130.62303
[14] Hansen, P. C., Rank-deficient and discrete ill-posed problems, (Ph.D. Thesis (1996), Technical University of Denmark)
[15] Helland, I. S., Relevance, Prediction and interpretation for linear methods with many predictors, (Research Report 0806-3842 (1996), Department of Mathematics, University of Oslo)
[16] Wold, S., A theoretical foundation of extra thermodynamic relationships (linear free energy relationships), Chimica Scripta, 5, 97-106 (1974)
[17] Helland, I. S., On the structure of partial least squares regression, Comm. Stat. Simulation Comput., 17, 2, 581-607 (1988) · Zbl 0695.62167
[18] Goutis, C., Partial least squares algorithm yields shrinkage estimators, Ann. Stat., 24, 2, 816-824 (1996) · Zbl 0859.62067
[19] Hoer, A. E.; Kennard, R. W., Ridge regression: Biased estimation for nonorthogonal problems, Technometrics, 12, 1, 55-109 (1970) · Zbl 0202.17205
[20] Fierro, R. D.; Bunch, J. R., Collinearity and total least squares, SIAM J. Matrix Anal. Appl., 15, 4, 1167-1181 (1994) · Zbl 0805.65042
[21] Priestley, M. B., Non-linear and Non-stationary Time Series Analysis (1988), Academic Press: Academic Press New York · Zbl 0667.62068
[22] Theiler, J., Estimating fractal dimension, J. Opt. Soc. Amer. A, 7, 6, 1055-1071 (1990)
[23] (Golub, G. H.; Van Loan, C. F., Matrix Computations (1989), Johns Hopkins University Press: Johns Hopkins University Press Baltimore, MD) · Zbl 0733.65016
[24] Hénon, M., A two-dimensional map with a strange attractor, Comm. Math. Phys., 50, 69-77 (1976) · Zbl 0576.58018
[25] Stewart, G. W., Perturbation theory for the singular value deomposition, (Vaccaro, R. J., SVD and Signal Processing II, Algorithms, Analysis and Applications (1991), Elsevier: Elsevier Amsterdam), 99-109
[26] Sauer, T.; Yorke, J. A.; Casdagli, M., Embedology, J. Stat. Phys., 65, 579-616 (1991) · Zbl 0943.37506
[27] Lorenz, E. N., Deterministic nonperiodic flow, J. Atmos. Sci., 20, 130 (1963) · Zbl 1417.37129
[28] Mackey, M.; Glass, L., Oscillation and chaos in physiological control systems, Science, 197, 287 (1977) · Zbl 1383.92036
[29] Ikeda, K., Multiple-valued stationary state and its instability of the transmitted light by a ring cavity system, Opt. Comm., 30, 257 (1979)
[30] Brown, R.; Bryant, P.; Abarbanel, H. D.I., Computing the Lyapunov spectrum of a dynamical system from an observed time series, Phys. Rev. A, 43, 6, 2787-2806 (1991)
[31] Farmer, J. D.; Sidorowich, J. J., Exploiting chaos to predict the future and reduce noise, (Lee, Y. C., Evolution, Learning and Cognition (1988), World Scientific: World Scientific Singapore), 277-330
[32] Casdagli, M., Nonlinear prediction of chaotic time series, Physica D, 35, 335-356 (1989) · Zbl 0671.62099
[33] (Goldberger, A. L., Clinical Electrocardiography (1977), Mosby: Mosby St. Louis)
[34] Skinner, J. E.; Goldberger, A. L.; Mayer-Kress, G.; Ideker, R. E., Chaos in the heart: Implications of clinical cardiology, Bio/Technology, 8, 1018 (1990)
[35] Kanters, J. K.; Holstein-Rathlou, N. H.; Agner, E., Lack of evidence for low-dimensional chaos in heart rate variability, J. Cardiovascular Electrophysiology, 5, 591 (1994)
[36] MIT-BIH Arrhythmia Database, (BMEC TR010 (1988), MIT: MIT Cambridge), (Revised), Biomedical Engineering Centre
[37] Lefebvre, J. H.; Goodings, D. A.; Kamath, M. V.; Fallen, E. L., Predictability of normal heart rythms and deterministic chaos, Chaos, 3, 2, 267-276 (1993)
[38] Sugihara, G.; May, R. M., Nonlinear forecasting as a way of distinguishing chaos from measurement error in time series, Nature, 344, 734-741 (1990)
[39] Tsonis, A. A.; Elsner, J. B., Global temperature as a regulator of climate predictability, Physica D, 1640, 1-7 (1997)
[40] Brillinger, D. R.; Guckenheimer, J.; Guttorp, P.; Oster, G., Empirical modelling of population time series data: The case of age and density dependent vital rates, (Lecture Notes in the Life Sci., 13 (1980)), 65-90 · Zbl 0451.92013
[41] Price, C. P.; Prichard, D.; Hogenson, E. A., Do the sunspot numbers form a “Chaotic” set?, J. Geophys. Res., 97, A12, 19113-119120 (1992)
[42] Carbonell, M.; Oliver, R.; Ballester, J. L., A search for chaotic behaviour in solar activity, Astronom. Astrophys., 290, 983-994 (1994)
[43] Mundt, M. D.; Maguire, W. B.; Chase, R. R.P., Chaos in the sunspot cycle: Analysis and prediction, J. Geophys. Res., 96, A2, 1705-1716 (1991)
[44] Tong, H., Non-linear Time Series: A Dynamical System Approach (1990), Oxford University Press: Oxford University Press New York
[45] Weigend, A. S.; Huberman, B. A.; Rumelhart, D. E., Predicting the future: A connectionist approach, Int. J. Neural Syst., 1, 3, 193-209 (1990)
[46] Lewis, P. A.W.; Stevens, J. G., Nonlinear modeling of time series using multivariate adaptive regression splines (MARS), J. Ameri. Stat. Assoc., 86, 416, 864-877 (1991) · Zbl 0850.62662
[47] Casdagli, M.; Jardins, D. D.; Eubank, S.; Farmer, J. D.; Gibson, J.; Theiler, J., Nonlinear modeling of chaotic time series: theory and applications, (Kim, J. H.K.; Stringer, J., Applied Chaos (1992), Wiley: Wiley New York), 335-380, Ch. 15
[48] Navone, H. D.; Ceccatto, H. A., Forecasting chaos from small data sets: A comparison of different nonlinear algorithms, J. Phys. A, 28, 12, 3381-3388 (1995) · Zbl 0860.58028
[49] Casdagli, M., Chaos and deterministic versus stochastic nonlinear modeling, J. Roy. Stat. Soc. Ser. B, 54, 303-328 (1992)
[50] Cheng, B.; Tong, H., On consistent nonparametric order determination and chaos, J. Roy. Stat. Soc. Ser. B, 54, 427-449 (1992) · Zbl 0782.62081
[51] Rössler, O. E., An equation for continuous chaos, Phys. Lett. A, 57, 397-398 (1976) · Zbl 1371.37062
[52] Kugiumtzis, D., State space reconstruction parameters in the analysis of chaotic time series — the role of the time window length, Physica D, 95, 13-28 (1996) · Zbl 0914.62063
[53] Smith, L. A., Local optimal prediction: Exploiting strangeness and the variation of sensitivity to initial condition, Philos. Trans. Roy. Soc. A, 348, 477-495 (1994)
[54] Sugihara, G., Nonlinear forecasting for the classification of natural time series, Philos. Trans. Roy. Soc. A, 348, 477-495 (1994) · Zbl 0864.92001
[55] Kostelich, E. J., Problems in estimating dynamics from the data, Physica D, 58, 138-152 (1992) · Zbl 1194.37134
[56] Jaeger, L.; Kantz, H., Unbiased reconstruction of the dynamics underlying a noisy chaotic time series, Chaos, 6, 440 (1996)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.