Intermittent estimation of stationary time series. (English) Zbl 1082.62073

Summary: Let \(\{X_n\}^\infty_{n=0}\) be a stationary real-valued time series with unknown distribution. Our goal is to estimate the conditional expectation of \(X_{n+1}\) based on the observations \(X_i\), \(0\leq i\leq n\), in a strongly consistent way. D. Bailey [Sequential schemes for classifying and predicting ergodic processes. Ph.D. thesis, Stanford Univ. (1976)] and B. Ya. Ryabko [Probl. Inf. Transm. 24, 87–96 (1988), translation from Probl. Peredachi Inf. 24, 3–14 (1988; Zbl 0666.94009)] proved that this is not possible even for ergodic binary time series if one estimates at all values of \(n\). We propose a very simple algorithm which will make predictions infinitely often at carefully selected stopping times chosen by our rule. We show that under certain conditions our procedure is strongly (pointwise) consistent, and \(L_2\) consistent without any condition. An upper bound on the growth of the stopping times is also presented in this paper.


62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62G05 Nonparametric estimation
62M20 Inference from stochastic processes and prediction
60G25 Prediction theory (aspects of stochastic processes)
60G10 Stationary stochastic processes


Zbl 0666.94009
Full Text: DOI arXiv


[1] Algoet, P. (1992). Universal schemes for prediction, gambling and portfolio selection.Annals of Probability, 20:901–941. · Zbl 0758.90006
[2] Algoet, P. (1994). The strong law of large numbers for sequential decisions under uncertainity.IEEE Transactions on Information Theory, 40:609–634. · Zbl 0827.62077
[3] Algoet, P. (1999). Universal schemes for learning the best nonlinear predictor given the infinite past and side information.IEEE Transactions on Information Theory, 45:1165–1185. · Zbl 0959.62078
[4] Ash, R. (1972).Real Analysis and Propbability Academic Press, New York. · Zbl 0249.28001
[5] Bailey, D. (1976).Sequential Schemes for Classifying and Predicting Ergodic Processes. Ph.D thesis, Stanford University.
[6] Cover, T. andThomas, J. (1991).Elements of Information Theory. Wiley, New York.
[7] Cover, T. M. (1975). Open problems in Information Theory, In1975 IEEE Joint workshop on Information Theory, pp. 35–36. IEEE Press, New York.
[8] Csiszár, I. (2002). Large-scale typicality of Markov sample paths and consistency of MDL order estimators.IEEE Transactons on Information Theory, 48:1616–1628. · Zbl 1060.62092
[9] Csiszár, I. andShields, P. (2000). The consistency of the BIC Markov order estimator.Annals of Statistics, 28:1601–1619. · Zbl 1105.62311
[10] Gray, R. (1988).Probability, Random Processes, and Ergodic Properties. Springer-Verlag, New York. · Zbl 0644.60001
[11] Györfi, L., Kohler, M., Krzyzak, A., andWalk.,H. (2002).A Distribution Free Theory of Nonparametric Regression. Springer-Verlag, New York.
[12] Györfi, L. andLugosi, G. (2002). Strategies for sequential prediction of stationary time series. In M. Dror, P. Ecuyer, and F. Szidarovszky eds.,Modeling Uncertainity An Examination of Stochastic Theory, Methods, and Applications, pp. 225–248. Kluwer Academic Publishers, Dordrecht.
[13] Györfi, L., Lugosi, G., andMorvai, G. (1999). A simple randomized algorithm for consistent sequential prediction of ergodic time series.IEEE Transactions on Information Theory, 45:2642–2650. · Zbl 0951.62080
[14] Györfi, L., Morvai, G., andYakowitz, S. (1998). Limits to consistent on-line forecasting for ergodic time series.IEEE Transactions on Information Theory, 44:886–892. · Zbl 0899.62122
[15] Kalikow, S. (1990). Random Markov processes and uniform martingales.Israel Journal of Mathematics, 71:33–54. · Zbl 0711.60041
[16] Keane, M. (1972). Strongly mixing g-measures.Invent. Math., 16:309–324. · Zbl 0241.28014
[17] Morvai, G. (2003). Guessing the output of a stationary, binary time series. In Y. Haitovsky, H. Lerche and Y. Ritov, eds.Foundations of Statistical Inference, pp. 205–213. Physika Verlag, Heidelberg New York. · Zbl 05280104
[18] Morvai, G. andWeiss, B. (2003). Forecasting for stationary binary time series.Acta Applicandae Mathematicae, 79:25–34. · Zbl 1030.62076
[19] Morvai, G., Yakowitz, S., andAlgoet, P. (1997). Weakly convergent nonparametric forecasting of stationary time series.IEEE Transactions on Information Theory, 43:483–498. · Zbl 0871.62082
[20] Morvai, G., Yakowitz, S., andGyörfi, L. (1996). Nonparametric inferences for ergodic, stationary time series.Annals of Statistics, 24:370–379. · Zbl 0855.62076
[21] Ornstein, D. (1974).Ergodic Theory, Randomness, and Dynamical Systems. Yale University Press, New Haven. · Zbl 0296.28016
[22] Ornstein, D. (1978). Guessing the next output of a stationary process.Israel Journal of Mathematics, 30:292–296. · Zbl 0386.60032
[23] Ornstein, D. andWeiss, B. (1993). Entropy and data compression schemes.IEEE Transactions on Information Theory, 39:78–83. · Zbl 0764.94003
[24] Révész, P. (1968).The Law of Large Numbers. Academic Press, New York. · Zbl 0203.50403
[25] Ryabko, B. Y. (1988). Prediction of random sequences and universal coding.Problems of Inform. Trans. 24:87–96. · Zbl 0666.94009
[26] Schäfer, D. (2002). Strongly consistent online forecasting of centered gaussian processes.IEEE Transactions on Information Theory, 48:791–799. · Zbl 1072.60501
[27] Shields, P. (1991). Cutting and stacking: a method for constructing stationary processes.IEEE Transactions on Information Theory, 37:1605–1614. · Zbl 0741.94007
[28] Weiss, B. (2000).Single Orbit Dynamics. American Mathematical Society, Providence, RI. · Zbl 1083.37500
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.