Dynamics of stochastic approximation algorithms. (English) Zbl 0955.62085

Azéma, Jacques (ed.) et al., Séminaire de probabilités XXXIII. Berlin: Springer. Lect. Notes Math. 1709, 1-68 (1999).
This is the author’s DEA course given at ENS Cachan during 1996-98. From the author’s abstract: The aim of this course is to introduce the reader to the dynamical system aspects of the theory of stochastic approximation.
The general form of stochastic approximation processes is \(x_{n+ 1}-x_n= \gamma_n(F(x_n) +U_n)\), \(n\in\mathbb{N}\). It is difficult to summarize in few lines all results given in this dense course; every section should be reviewed separately. The basic concept is “asymptotic pseudotrajectories”, whose definition is given in section 3. All other sections give results about the link between asymptotic pseudotrajectories and stochastic approximation.
From the author’s introduction: In section 4 classical results on stochastic approximations are (re)formulated in the language of asymptotic pseudotrajectories. It is shown that, under suitable conditions, the continuous time process obtained by a convenient interpolation of \(\{x_n\}\) is almost surely an asymptotic pseudotrajectory of the semiflow induced by the associated ODE \(dx/dt=F(x)\dots\)
Section 5 characterizes the limit sets of asymptotic pseudotrajectories. The main result of the section establishes that limit sets of precompact asymptotic pseudotrajectories are internally chain-transitive…
Section 6 applies the abstract results of section 5 in various situations. It is shown how assumptions on the deterministic dynamics can help to identify the possible limit sets of stochastic approximation processes with a great deal of generality.…
Section 7 establishes simple sufficient conditions ensuring that a given attractor of an ODE has a positive probability to host the limit set of the stochastic approximation process…
Section 8 considers the question of shadowing…Section 9 focusses on the behavior of stochastic approximation processes near ‘unstable’ sets…
Section 10 introduces the notion of a stochastic process being a weak asymptotic pseudodotrajectory for a semiflow and analyzes properties of its empirical occupation measure.
For the entire collection see [Zbl 0924.00016].


62L20 Stochastic approximation
37L30 Attractors and their dimensions, Lyapunov exponents for infinite-dimensional dissipative dynamical systems
Full Text: Numdam