On controlled Markov processes with average cost criterion. (English) Zbl 0633.90090

Mandl, Petr; Romera Ayllon, M. Rosario

Kybernetika 23, 433-442 (1987).

The authors consider continuous-time controlled Markov chains with costs in finite time equal to the integral of a function of the path and control plus a function evaluated at jumps. A stationary control is sought that minimizes the long-term time-average cost. They prove a theorem on the normal limit of the finite-time cost as the time becomes large, and show that the time spent by that cost above its asymptotic mean has under certain conditions an arcsine distribution.

Reviewer: K.Wickwire

MSC:

90C40

Markov and semi-Markov decision processes

Keywords:

average cost criterion; continuous-time controlled Markov chains; finite time; stationary control; arcsine distribution

References:

[1]	P. Billingsley: Convergence of Probability Measures. J. Wiley, New York 1968. · Zbl 0172.21201
[2]	P. Mandl: Martingale methods in discrete state random processes. Kybernetika 18 (1982), supplement.
[3]	P. Mandl: Limit theorems of probability theory and optimality in linear controlled systems with quadratic cost. Proceedings of 5th IFIP Working Conference on Stochastic Differential Systems. (Lecture Notes in Control and Information Sciences 96.) Springer-Verlag, Berlin 1987,316-329. · Zbl 0637.93078
[4]	M. R. Romera Ayllón: Control adaptivo de procesos de Markov con espacio de estados numerable. Thesis. Universidad Complutense, Madrid 1984.
[5]	A. Wald: Sequential Analysis. J. Wiley, New York 1947. · Zbl 0041.26303