×
Schweitzer, Paul J.

On the existence of relative values for undiscounted Markovian decision processes with a scalar gain rate. (English) Zbl 0598.90091

J. Math. Anal. Appl. 104, 67-78 (1984).
The functional equations \(v=\max \{q(f)-gT(f)+P(f)v\); \(f\in K\}\equiv Qv\) of undiscounted semi-Markovian decision processes are shown to be solvable if and only if all components of the maximum gain rate vector are equal. More generally, in the multichain case, the functional equations for the value vector possess a solution if and only if there is a policy which achieves the maximal gain vector. The method of proof exhibits vectors \(v^{\pm}\) such that \(Qv^+\leq v^+\) and \(Qv^-\geq v^-\).

Cited in 3 Documents

MSC:

90C40 Markov and semi-Markov decision processes

Keywords:

undiscounted semi-Markovian decision processes; maximum gain rate vector
PDF BibTeX XML Cite
\textit{P. J. Schweitzer}, J. Math. Anal. Appl. 104, 67--78 (1984; Zbl 0598.90091)
Full Text: DOI
  • logo of hbz
  • logo of WorldCat

References:

[1] Bather, J, Optimal decision procedures for finite Markov chains. part II. communicating systems, Adv. in appl. probab., 5, 521-540, (1971) · Zbl 0275.90049
[2] Denardo, E; Fox, B, Multichain Markov renewal programs, SIAM J. appl. math., 16, 468-487, (1968) · Zbl 0201.19303
[3] Doob, J, Stochastic processes, (1953), Wiley New York
[4] Edwards, R.E, Functional analysis: theory and applications, (1965), Holt, Rinehart & Winston New York · Zbl 0182.16101
[5] Fox, B, (g, w) optima in Markov renewal programs, Management sci., 15, 210-212, (1968)
[6] Gale, D, The theory of linear economic models, (1960), McGraw-Hill New York · Zbl 0114.12203
[7] Howard, R.A, Semi-Markovian decision processes, Bull. internat. statist. inst., 40, 625-652, (1963), Part 2 · Zbl 0128.12804
[8] Jewell, W.S, Markov renewal programming, I and II, Oper. res., 11, 938-971, (1963) · Zbl 0126.15905
[9] Kemeny, J.G; Snell, J.L, Finite Markov chains, (1963), Van Nostrand Princeton, N. J · Zbl 0115.13604
[10] Romanovsky, I.V, On the solvability of Bellman’s functional equation for a Markovian decision process, J. math. anal. appl., 42, 485-498, (1973) · Zbl 0263.90041
[11] Schweitzer, P.J, Perturbation theory and undiscounted Markov renewal programming, Oper. res., 17, 716-727, (1969) · Zbl 0176.50003
[12] Schweitzer, P.J; Federgruen, A, Functional equations of undiscounted Markov renewal programming, Math. oper. res., 3, 308-322, (1978) · Zbl 0388.90083
[13] Schweitzer, P.J, On the solvability of Bellman’s functional equations for Markov renewal programming, J. math. anal. appl., 96, 13-23, (1983) · Zbl 0526.90094
[14] Sheu, S.S; Farn, K.-J, A sufficient condition for the existence of a stationary 1-optimal plan in compact action Markovian decision processes, (), 111-126
[15] Veinott, A.F, On finding optimal policies in discrete dynamic programming with no discounting, Ann. math. statist., 37, 1284-1294, (1966) · Zbl 0149.16301
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.