Occupation measures for controlled Markov processes: Characterization and optimality. (English) Zbl 0863.93086

Bhatt, Abhay G.; Borkar, Vivek S.

Ann. Probab. 24, No. 3, 1531-1562 (1996).

Authors’ summary: For controlled Markov processes taking values in a Polish space, control problems with ergodic cost, infinite-horizon discounted cost and finite-horizon cost are studied. Each is posed as a convex optimization problem wherein one tries to minimize a linear functional on a closed convex set of appropriately defined occupation measures for the problem. These are characterized as solutions of a linear equation associated with the problem. This characterization is used to establish the existence of optimal Markov controls. The dual convex optimization problem is also studied.

Reviewer: W.Kliemann (Ames)

MSC:

93E20	Optimal stochastic control
60J25	Continuous-time Markov processes on general state spaces

Keywords:

optimal controls; controlled Markov processes; ergodic cost; discounted cost; convex optimization; dual convex optimization