×

Modeling of time series using random forests: theoretical developments. (English) Zbl 1454.62256

The authors apply random forests methodology to nonlinear time series modeling. The paper provides theoretical justification for applications in time series settings. A uniform concentration inequality across all regression trees built on nonlinear autoregressive processes is obtained under a mild condition on their minimum leaf size. Then this result is used to prove consistency for a large class of random forests. The approach is based on the theory of Markov processes and Bernstein type concentration inequalities. The assumptions are explicit in terms of a suitable smooth regression function \(f\) and the distribution of the noise term. For example, they are applicable if \(f\) is bounded and Lipschitz continuous, and the noise sequence is i.i.d. with a light-tailed distribution. Simulation studies for various specifications of \(f\) are presented.

MSC:

62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62G05 Nonparametric estimation
62G08 Nonparametric regression and quantile regression
62M05 Markov processes: estimation; hidden Markov models
62H30 Classification and discrimination; cluster analysis (statistical aspects)
60G10 Stationary stochastic processes
60J05 Discrete-time Markov processes on general state spaces

Software:

GeneSrF
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Adams, T. M. and Nobel, A. B. (2010). Uniform convergence of Vapnik-Chervonenkis classes under ergodic sampling., Ann. Probab. 38 1345-1367. · Zbl 1220.60019 · doi:10.1214/09-AOP511
[2] Amaratunga, D., Cabrera, J. and Lee, Y.-S. (2008). Enriched random forests., Bioinformatics 24 2010-2014.
[3] An, H. Z. and Huang, F. C. (1996). The geometrical ergodicity of nonlinear autoregressive models., Statist. Sinica 6 943-956. · Zbl 0857.62085
[4] Arlot, S. and Genuer, R. (2014). Analysis of purely random forests bias., arXiv preprint arXiv:1407.3939. · Zbl 1402.62131 · doi:10.1007/s11749-016-0484-4
[5] Biau, G. (2012). Analysis of a random forests model., J. Mach. Learn. Res. 13 1063-1095. · Zbl 1283.62127
[6] Biau, G. and Devroye, L. (2010). On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification., J. Multivariate Anal. 101 2499-2518. · Zbl 1198.62048 · doi:10.1016/j.jmva.2010.06.019
[7] Biau, G. and Scornet, E. (2016). A random forest guided tour., TEST 25 197-227. · Zbl 1402.62133 · doi:10.1007/s11749-016-0481-7
[8] Breiman, L. (1996). Bagging predictors., Machine learning 24 123-140. · Zbl 0858.68080
[9] Breiman, L. (2001). Random forests., Machine learning 45 5-32. · Zbl 1007.68152 · doi:10.1023/A:1010933404324
[10] Breiman, L., Friedman, J. H., Olshen, R. A. and Stone, C. J. (1984)., Classification and regression trees. Wadsworth Statistics/Probability Series. Wadsworth Advanced Books and Software, Belmont, CA. · Zbl 0541.62042
[11] Cutler, D. R., Edwards Jr, T. C., Beard, K. H., Cutler, A., Hess, K. T., Gibson, J. and Lawler, J. J. (2007). Random forests for classification in ecology., Ecology 88 2783-2792.
[12] De la Pena, V. and Giné, E. (1999)., Decoupling: From Dependence to Independence. Springer Science & Business Media.
[13] Díaz-Uriarte, R. and De Andres, S. A. (2006). Gene selection and classification of microarray data using random forest., BMC bioinformatics 7 3.
[14] Doukhan, P. (2012)., Mixing: properties and examples 85. Springer Science & Business Media.
[15] Genuer, R. (2012). Variance reduction in purely random forests., J. Nonparametr. Stat. 24 543-562. · Zbl 1254.62050 · doi:10.1080/10485252.2012.677843
[16] Geurts, P., Ernst, D. and Wehenkel, L. (2006). Extremely randomized trees., Machine learning 63 3-42. · Zbl 1110.68124 · doi:10.1007/s10994-006-6226-1
[17] Gu, S., Kelly, B. and Xiu, D. (2020). Empirical asset pricing via machine learning., The Review of Financial Studies 33 2223-2273.
[18] Härdle, W., Lütkepohl, H. and Chen, R. (1997). A review of nonparametric time series analysis., International Statistical Review 65 49-72. · Zbl 0887.62043
[19] Howard, J. and Bowles, M. (2012). The two most important algorithms in predictive modeling today. In, Strata Conference presentation, February 28.
[20] Kumar, M. and Thenmozhi, M. (2006). Forecasting stock index movement: A comparison of support vector machines and random forest. In, Indian institute of capital markets 9th capital markets conference paper.
[21] Lin, Y. and Jeon, Y. (2006). Random forests and adaptive nearest neighbors., J. Amer. Statist. Assoc. 101 578-590. · Zbl 1119.62304 · doi:10.1198/016214505000001230
[22] Meinshausen, N. (2006). Quantile regression forests., J. Mach. Learn. Res. 7 983-999. · Zbl 1222.68262
[23] Merlevède, F., Peligrad, M. and Rio, E. (2009). Bernstein inequality and moderate deviations under strong mixing conditions. In, High dimensional probability V: the Luminy volume. Inst. Math. Stat. (IMS) Collect. 5 273-292. Inst. Math. Statist., Beachwood, OH. · Zbl 1243.60019
[24] Prasad, A. M., Iverson, L. R. and Liaw, A. (2006). Newer classification and regression tree techniques: bagging and random forests for ecological prediction., Ecosystems 9 181-199.
[25] Rio, E. (1993). Covariance inequalities for strongly mixing processes., Ann. Inst. H. Poincaré Probab. Statist. 29 587-597. · Zbl 0798.60027
[26] Scornet, E., Biau, G. and Vert, J.-P. (2015). Consistency of random forests., Ann. Statist. 43 1716-1741. · Zbl 1317.62028 · doi:10.1214/15-AOS1321
[27] Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A. and Blake, A. (2011). Real-time human pose recognition in parts from single depth images. In, CVPR 2011 1297-1304. IEEE.
[28] Wager, S. and Athey, S. (2018). Estimation and inference of heterogeneous treatment effects using random forests., J. Amer. Statist. Assoc. 113 1228-1242. · Zbl 1402.62056 · doi:10.1080/01621459.2017.1319839
[29] Wager, S.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.