A Bernstein-type inequality for some mixing processes and dynamical systems with an application to learning. (English) Zbl 1388.60060

Hang, Hanyuan; Steinwart, Ingo

Ann. Stat. 45, No. 2, 708-743 (2017).

Authors’ abstract: The authors establish a Bernstein-type inequality for a class of stochastic processes that includes the classical geometrically \(\varphi\)-mixing processes, Rio’s generalization of these processes and many time-discrete dynamical systems. Modulo a logarithmic factor and some constants, the derived Bernstein-type inequality coincides with the classical Bernstein inequality for i.i.d. data. The new Bernstein-type inequality is further used to derive an oracle inequality for generic regularized empirical risk minimization algorithms and data generated by such processes. Applying this oracle inequality to support vector machines using the Gaussian kernels for binary classification, the same rate as for i.i.d. processes, and for least squares and quantile regression is obtained; it turns out that the resulting learning rates match, up to some arbitrarily small extra term in the exponent, the optimal rates for i.i.d. processes.

Reviewer: Giovanni Puccetti (Milano)

MSC:

60E15	Inequalities; stochastic orderings
60G10	Stationary stochastic processes
37D20	Uniformly hyperbolic systems (expanding, Anosov, Axiom A, etc.)
60F10	Large deviations
68T05	Learning and adaptive systems in artificial intelligence
62G08	Nonparametric regression and quantile regression
62M10	Time series, auto-correlation, regression, etc. in statistics (GARCH)

Keywords:

Bernstein-type inequalities; mixing processes; dynamical systems; nonparametric classification and regression; support vector machines (SVMs)