# zbMATH — the first resource for mathematics

Impulsive and continuously acting control of jump processes – time discretization. (English) Zbl 0739.60076
Summary: We deal with a model of controlled Markovian jump processes. Two kinds of controls are admitted: on the one hand “generator controls” (“G- controls”), affecting the jump intensity of the process, on the other hand “impulsive controls” (“I-controls”), causing immediate jumps. The controls are allowed to be randomized and history-dependent. Essential results are obtained by the technique of time-discretization: a family ($$\Delta^ h)_{h>0}$$ of discrete time models is constructed, approximating a given continuous time model $$\Gamma$$ in an appropriate manner. We give conditions yielding the convergence of the family $$(v^ h)_{h>0}$$, where $$v^ h$$ is the optimal value function of $$\Delta^ h$$, to the optimal value function of $$\Gamma$$. This result helps us to construct optimal Markov policies. The problem of constructing $$\varepsilon$$-optimal policies by extension of certain $$\Delta^ h$$- policies is treated. Finally, examples will be given.

##### MSC:
 60J75 Jump processes (MSC2010) 90C40 Markov and semi-Markov decision processes 93E20 Optimal stochastic control 60J05 Discrete-time Markov processes on general state spaces 90B05 Inventory, storage, reservoirs 90B25 Reliability, availability, maintenance, inspection in operations research
Full Text: