Ergodic theorems for discrete time stochastic systems using a stochastic Lyapunov function. (English) Zbl 0681.60067

Summary: Sufficient conditions are established under which the law of large numbers and related ergodic theorems hold for nonlinear stochastic systems operating under feedback. It is shown that these conditions hold whenever a moment condition is satisfied; this may be interpreted as a generalization of the martingale property.
If, in addition, a stochastic controllability condition holds, then it is shown that the underlying distributions governing the system converge to an invariant probability at a geometric rate. The key assumption used is that a Markov chain with stationary transition probabilities exists that serves as a state process for the closed loop system.


60J20 Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.)
93E15 Stochastic stability in control theory
60J10 Markov chains (discrete-time Markov processes on discrete state spaces)
Full Text: DOI