zbMATH — the first resource for mathematics

Limits to classification and regression estimation from ergodic processes. (English) Zbl 0933.62033
Summary: We answer two open questions concerning the existence of universal schemes for classification and regression estimation from stationary ergodic processes. It is shown that no measurable procedure can produce weakly consistent regression estimates from every bivariate stationary ergodic process, even if the covariate and response variables are restricted to take values in the unit interval. It is further shown that no measurable procedure can produce weakly consistent classification rules from every bivariate stationary ergodic process for which the response variable is binary valued. The results of the paper are derived via reduction arguments and are based in part on recent work concerning density estimation from ergodic processes.

62G08 Nonparametric regression and quantile regression
62M99 Inference from stochastic processes
62G07 Density estimation
60G10 Stationary stochastic processes
Full Text: DOI
[1] Adams, T. M. (1997). Families of ergodic processes without consistent density or regression estimates.
[2] Adams, T. M. and Nobel, A. B. (1998). On density estimation from ergodic processes. Ann. Probab. 26 794-804. · Zbl 0932.62042
[3] Algoet, P. (1992). Universal schemes for prediction, gambling and portfolio selection. Ann. · Zbl 0758.90006
[4] Probab. 20 901-941. Correction: [Ann. Probab. 23 474-478 (1995).] · Zbl 0837.90011
[5] Ash, R. (1972). Real Analysis and Probability. Academic Press, New York. · Zbl 0249.28001
[6] Bailey, D. H. (1976). Sequential schemes for classifying and predicting ergodic processes. Ph.D. dissertation, Dept. Mathematics, Stanford Univ.
[7] Barron, A. R. (1985). Logically smooth density estimation. Technical Report TR 56, Dept. Statistics, Stanford Univ.
[8] Cheng, B. C. and Robinson, P. M. (1991). Density estimation in strongly dependent non-linear time series. Statist. Sinica 1 335-359. · Zbl 0823.62031
[9] Delecroix, M. (1987). Sur l’estimation et la prévision non-paramétrique des processus ergodiques. Ph.D. thesis, Univ. Lille Flandres Artois, Lille, France.
[10] Delecroix, M. and Rosa, A. C. (1996). Nonparametric estimation of a regression function and its derivatives under an ergodic hypothesis. J. Nonparametric Statist. 6 367-382. · Zbl 0879.62034
[11] Devroye, L. and Wagner, T. (1980). Distribution-free consistency results in nonparametric discrimination and regression function estimation. Ann. Statist. 8 231-239. · Zbl 0431.62025
[12] Devroye, L., Gy örfi, L. and Lugosi, G. (1996). A Probabalistic Theory of Pattern Recognition. Springer, New York. · Zbl 0853.68150
[13] Gy örfi, L. (1981). Strongly consistent density estimate from ergodic sample. J. Multivariate Anal. 11 81-84. · Zbl 0449.62031
[14] Gy örfi, L., Härdle, W., Sarda, P. and Vieu. P. (1989). Nonparametric Curve Estimation from Time Series. Springer, Berlin. · Zbl 0697.62038
[15] Gy örfi, L. and Lugosi, G. (1992). Kernel density estimation from ergodic sample is not universally consistent. Comput. Statist. Data Anal. 14 437-442. · Zbl 0937.62580
[16] Gy örfi, L., Morvai, G. and Yakowitz, S. (1998). Limits to consistent on-line forecasting for ergodic time series. IEEE Trans. Inform. Theory 44 886-892. · Zbl 0899.62122
[17] Hidalgo, J. (1997). Non-parametric estimation with strongly dependent time multivariate time series. J. Time Ser. Anal. 18 95-122. · Zbl 0923.62045
[18] Ho, H. C. (1995). On the strong uniform consistency of density estimation for strongly dependent sequences. Statist. Probab. Lett. 22 149-156. · Zbl 0810.62039
[19] Masry, E. (1996). Multivariate local polynomial regression for time series: uniform strong consistency and rates. J. Time Ser. Anal. 17 571-599. · Zbl 0876.62075
[20] Morvai, G., Yakowitz, S. and Algoet, P. (1997). Weakly convergent nonparametric forecasting of stationary time series. IEEE Trans. Inform. Theory 43 483-498. · Zbl 0871.62082
[21] Morvai, G., Yakowitz, S. and Gy örfi, L. (1996). Nonparametric inference for ergodic, stationary time series. Ann. Statist. 24 370-379. · Zbl 0855.62076
[22] Morvai, G., Kulkarni, S. and Nobel, A. B. (1997). Regression estimation from an individual stationary sequence. Unpublished manuscript. · Zbl 0939.62034
[23] Nobel, A. B., Morvai, G. and Kulkarni, S. (1997). Density estimation from an individual numerical sequence. IEEE Trans. Inform. Theory 44 537-541. · Zbl 0905.62037
[24] Ornstein, D. S. (1978). Guessing the next output of a stationary process. Israel J. Math. 30 292-296. · Zbl 0386.60032
[25] Rosenblatt, M. (1970). Density estimates and Markov sequences. In Nonparametric Techniques in Statistical Inference (M. Puri, ed.) 199-210. Cambridge Univ. Press, London.
[26] Rosenblatt, M. (1991). Stochastic Curve Estimation. IMS, Hayward, CA. · Zbl 1163.62318
[27] Roussas, G. (1967). Nonparametric estimation in Markov processes. Ann. Inst. Statist. Math. 21 73-87. · Zbl 0181.45804
[28] Roussas, G. (1969). Nonparametric estimation of the transition distribution function of a Markov process. Ann. Math. Statist. 40 1386-1400. · Zbl 0188.50501
[29] Ryabko, B. Ya. (1988). Prediction of random sequences and universal coding. Problems Inform. Transmission 24 87-96. · Zbl 0666.94009
[30] Spiegelman, C. and Sacks, J. (1980). Consistent window estimation in nonparametric regression. Ann. Statist. 8 240-246. · Zbl 0432.62066
[31] Stone, C. (1977). Consistent nonparametric regression. Ann. Statist. 5 595-620. · Zbl 0366.62051
[32] Yakowitz, S. (1993). Nearest neighbor regression estimation for null-recurrent Markov time series. Stochastic Process. Appl. 48 311-318. · Zbl 0791.62084
[33] Yakowitz, S., Gy örfi, L., Kieffer, J. and Morvai, G. (1997). Strongly consistent nonparametric estimation of smooth regression functions for stationary ergodic sequences. J. Multivariate Anal. · Zbl 1070.62518
[34] Yakowitz, S. and Heyde, C. (1997). Long range dependency effects with implications for forecasting and queueing inference. Unpublished manuscript.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.