Random forests for time-dependent processes.

*(English)*Zbl 1455.62172Summary: Random forests were introduced by L. Breiman [Mach. Learn. 45, No. 1, 5–32 (2001; Zbl 1007.68152)]. We study theoretical aspects of both original Breiman’s random forests and a simplified version, the centred random forests. Under the independent and identically distributed hypothesis, E. Scornet et al. [Ann. Stat. 43, No. 4, 1716–1741 (2015; Zbl 1317.62028)] proved the consistency of Breiman’s random forest, while G. Biau [J. Mach. Learn. Res. 13, 1063–1095 (2012; Zbl 1283.62127)] studied the simplified version and obtained a rate of convergence in the sparse case. However, the i.i.d hypothesis is generally not satisfied for example when dealing with time series. We extend the previous results to the case where observations are weakly dependent, more precisely when the sequences are stationary \(\beta\)-mixing.

##### MSC:

62M10 | Time series, auto-correlation, regression, etc. in statistics (GARCH) |

62H30 | Classification and discrimination; cluster analysis (statistical aspects) |

68T05 | Learning and adaptive systems in artificial intelligence |

62G05 | Nonparametric estimation |

62G20 | Asymptotic properties of nonparametric inference |

60G12 | General second-order stochastic processes |

Full Text:
DOI

##### References:

[1] | H.C.P. Berbee, Random walks with stationary increments and renewal theory. MC Tracts 112 (1979) 1-223. · Zbl 0443.60083 |

[2] | G. Biau, Analysis of a random forests model. J. Mach. Learn. Res. 13 (2012) 1063-1095. · Zbl 1283.62127 |

[3] | G. Biau and E. Scornet, A random forest guided tour. TEST 25 (2016) 197-227. · Zbl 1402.62133 |

[4] | R.C. Bradley, Basic properties of strong mixing conditions. a survey and some open questions. Probab. Surv. 2 (2005) 107-144. · Zbl 1189.60077 |

[5] | L. Breiman, Bagging predictors. Mach. Learn. 24 (1996) 123-140. · Zbl 0858.68080 |

[6] | L. Breiman, Random forests. Mach. Learn. 45 (2001) 5-32. · Zbl 1007.68152 |

[7] | L. Breiman, Consistency for a simple model of random forests. Technical report (2004). |

[8] | L. Breiman, J. Friedman, C.J. Stone and R.A. Olshen, Classification and Regression Trees. The Wadsworth and Brooks-Cole statistics-probability series. Taylor & Francis, Oxford (1984). |

[9] | D.R. Cutler, T.C. Edwards, K.H. Beard, A. Cutler, K.T. Hess, J. Gibson and J.J. Lawler, Random forests for classification in ecology. Ecology 88 (2007) 2783-2792. |

[10] | J. Dedecker, P. Doukhan, G. Lang, L.R.J. Rafael, S. Louhichi and C. Prieur, Weak dependence, in Weak Dependence: With Examples and Applications. Springer, Berlin (2007) 9-20. · Zbl 1165.62001 |

[11] | G. Dudek, Short-term load forecasting using random forests, in Intelligent Systems’2014. Springer International Publishing, Cham (2015) 821-828. |

[12] | A. Fischer, L. Montuelle, M. Mougeot and D. Picard, Statistical learning for wind power: A modeling and stability study towards forecasting. Wind Energy 20 (2017) 2037-2047. |

[13] | L. Györfi, M. Kohler, A. Krzyzak and H. Walk, A distribution-free theory of nonparametric regression. Springer Science & Business Media, Berlin (2006). · Zbl 1021.62024 |

[14] | M.J. Kane, N. Price, M. Scotch and P. Rabinowitz, Comparison of arima and random forest time series models for prediction of avian influenza h5n1 outbreaks. BMC Bioinform. 15 (2014) 276. |

[15] | A. Lahouar and J. Ben Hadj Slama, Random forests model for one day ahead load forecasting, in IREC2015 The Sixth International Renewable Energy Congress (2015) 1-6. |

[16] | A.C. Lozano, S.R. Kulkarni and R.E. Schapire, Convergence and consistency of regularized boosting with weakly dependent observations. IEEE Trans. Inf. Theory 60 (2014) 651-660. · Zbl 1364.94211 |

[17] | R. Meir, Nonparametric time series prediction through adaptive model selection. Mach. Learn. 39 (2000) 5-34. · Zbl 0954.68124 |

[18] | L. Mentch and G. Hooker, Quantifying uncertainty in random forests via confidence intervals and hypothesis tests. J. Mach. Learn. Res. 17 (2016) 1-41. · Zbl 1360.62095 |

[19] | A.M. Prasad, L.R. Iverson and A. Liaw, Newer classification and regression tree techniques: bagging and random forests for ecological prediction. Ecosystems 9 (2006) 181-199. |

[20] | E. Rio, Inequalities and limit theorems for weakly dependent sequences. Lecture (2013). |

[21] | E. Scornet, On the asymptotics of random forests. J. Multivar. Anal. 146 (2016) 72-83. · Zbl 1337.62063 |

[22] | E. Scornet, G. Biau and J.-P. Vert, Consistency of random forests. Ann. Stat. 43 (2015) 1716-1741. · Zbl 1317.62028 |

[23] | J. Shotton, T. Sharp, A. Kipman, A. Fitzgibbon, M. Finocchio, A. Blake, M. Cook and R. Moore, Real-time human pose recognition in parts from single depth images. Commun. ACM 56 (2013) 116-124. |

[24] | V. Svetnik, A. Liaw, C. Tong, J.C. Culberson, R.P. Sheridan and B.P. Feuston, Random forest: a classification and regression tool for compound classification and qsar modeling. J. Chem. Inf. Comput. Sci. 43 (2003) 1947-1958. |

[25] | S. Wager and S. Athey, Estimation and inference of heterogeneous treatment effects using random forests. J. Am. Stat. Assoc. 113 (2018) 1228-1242. · Zbl 1402.62056 |

[26] | B. Yu, Rates of convergence for empirical processes of stationary mixing sequences. Ann. Prob. 22 (1994) 94-116. · Zbl 0802.60024 |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.