Gittins indices in the dynamic allocation problem for diffusion processes. (English) Zbl 0536.60058

At each instant t one is allowed to choose a single project \(j=i(t)\in \{1,2,...,d\}\), which then evolves as a diffusion with local drift \(\mu_ j(x)\) and variance \(\sigma^ 2_ j(x)\). The stochastic control problem considered by the author is to maximize the expected discounted reward \(E\int^{\infty}_{0}e^{-at}h(i(t),x_{i(t)}(t))dt\). He presents a rigorous discussion of this problem and derives the optimal ”allocation policy” \(\{i(t),t\geq 0\}\) in terms of so-called Gittins indices. Very explicit computations of the index are offered.
Reviewer: V.Mackevičius


60G40 Stopping times; optimal stopping problems; gambling theory
93E20 Optimal stochastic control
60J60 Diffusion processes
Full Text: DOI