zbMATH — the first resource for mathematics

Impulsive and continuously acting control of jump processes – time discretization. (English) Zbl 0739.60076
Summary: We deal with a model of controlled Markovian jump processes. Two kinds of controls are admitted: on the one hand “generator controls” (“G- controls”), affecting the jump intensity of the process, on the other hand “impulsive controls” (“I-controls”), causing immediate jumps. The controls are allowed to be randomized and history-dependent. Essential results are obtained by the technique of time-discretization: a family (\(\Delta^ h)_{h>0}\) of discrete time models is constructed, approximating a given continuous time model \(\Gamma\) in an appropriate manner. We give conditions yielding the convergence of the family \((v^ h)_{h>0}\), where \(v^ h\) is the optimal value function of \(\Delta^ h\), to the optimal value function of \(\Gamma\). This result helps us to construct optimal Markov policies. The problem of constructing \(\varepsilon\)-optimal policies by extension of certain \(\Delta^ h\)- policies is treated. Finally, examples will be given.

60J75 Jump processes (MSC2010)
90C40 Markov and semi-Markov decision processes
93E20 Optimal stochastic control
60J05 Discrete-time Markov processes on general state spaces
90B05 Inventory, storage, reservoirs
90B25 Reliability, availability, maintenance, inspection in operations research
Full Text: DOI