Perturbation theory for Markov reward processes with applications to queueing systems. (English) Zbl 0642.60100

van Dijk, Nico M.; Puterman, Martin L.

Adv. Appl. Probab. 20, No. 1, 79-98 (1988).

Consider a discrete time, discrete state Markov chain where a bounded reward r(i) is obtained every time when the chain enters state i. The problem: If the transition probabilities and/or the reward function are perturbated to a certain amount, what will the change of the finite- horizon reward, the discounted infinite-horizon reward, the average reward and the total reward up to a random time be? Some easy to verify conditions are given which make the theorems concerned with the above questions better suited for applications e.g. in queueing theory.

Reviewer: H.Daduna

MSC:

60K30	Applications of queueing theory (congestion, allocation, storage, traffic, etc.)
90C47	Minimax problems in mathematical programming
90B22	Queues and service in operations research
90C31	Sensitivity, stability, parametric optimization

Keywords:

perturbation theory; Markov decision processes; reward function