zbMATH — the first resource for mathematics

Necessary and sufficient optimality conditions for control of piecewise deterministic Markov processes. (English) Zbl 0762.93080
Summary: Piecewise deterministic Markov processes (PDPs) are continuous time homogeneous Markov processes whose trajectories are solutions of ordinary differential equations with random jumps between the different integral curves. Both continuous deterministic motion and the random jumps of the processes are controlled in order to minimize the expected value of a performance criterion involving discounted running and boundary costs. Under fairly general assumptions, we show that there exists an optimal control, that the value function is Lipschitz continuous and that a generalized Bellman-Hamilton-Jacobi equation involving the Clarke generalized gradient is a necessary and sufficient condition for the problem.

93E20 Optimal stochastic control
Full Text: DOI