# zbMATH — the first resource for mathematics

Monotonicity in a Markov decision process. (English) Zbl 0646.90088
Consider the Markov decision process with state space $$I=\{0,...,n\}$$ and closed action set $$A\subseteq [0,\infty]$$, having optimality equation $V_ i=\min_{a}\{a+c+\int^{i}_{j=0}p(i,j,a)V_ j\},\quad 1\leq i\leq n,\quad V_ 0=0,$ in which the transition probabilities are given by $p(i,j,a)=\left( \begin{matrix} i\\ j\end{matrix} \right)p^{aj}(1-p^ a)^{i-j},$ 0$$<p<1$$ and $$c>0$$. The model can be interpreted in terms of collective sequential testing of n subjects. When $$A=[0,\infty]$$ it is shown that $$V_ i$$ is increasing and concave in i, and that the optimal actions $$a_ i$$ are strictly increasing in i. However, when A is the set of positive integers, whilst the same properties hold for $$V_ i$$, monotonicity of $$a_ i$$ is more difficult to establish: a restriction on the values of the parameters p and c is given as a sufficient condition.
Reviewer: J.Preater

##### MSC:
 90C40 Markov and semi-Markov decision processes
Full Text: