On adaptive control of Markov processes. (English) Zbl 0639.60049

This paper extends the results of the first author’s previous work, Probability theory, Warsaw 1976, Banach Cent. Publ. 5, 159-175 (1979; Zbl 0439.60069), to the continuous-time case. A sufficient condition to ensure the existence of an optimal stationary strategy to control a countable Markov process is given. The asymptotic behaviour of the criterion functional is investigated. An example is given.
Reviewer: M.Tibaldi


60G35 Signal detection and filtering (aspects of stochastic processes)
93E20 Optimal stochastic control


Zbl 0439.60069
Full Text: EuDML


[1] B. M. Brown: Martingale Central Limit Theorems. Ann. Math. Statist. 42 (1971), 59-66. · Zbl 0218.60048
[2] A. Hordijk: Dynamic Programming and Markov Potential Theory. Math. Centrum, Amsterdam 1974. · Zbl 0284.49012
[3] P. Mandl: On the adaptive control of countable Markov chains. Prob. Theory, Banach Center Publications, Vol. 5, 159-173, Warsaw 1979. · Zbl 0439.60069
[4] P. Mandl: Martingale Methods in Discrete State Random Processes. Supplement to the Journal Kybernetika 18 (1982).
[5] M. R. Romera: Adaptive Control of Markov Processes with Countable State Space. Doctoral Thesis (in Spanish). Universidad Complutense, Madrid 1985.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.