Controlled semi-Markov models - the discounted case. (English) Zbl 0673.93089

Bhattacharya, Rabi N.; Majumdar, Mukul

J. Stat. Plann. Inference 21, No. 3, 365-381 (1989).

Summary: Let the state space S be a Borel subset of a complete separable metric space, the action space A compact metric. Existence of stationary optimal policies is proved for general semi-Markov models with possibly unbounded rewards. The corresponding dynamic programming equations are also derived. The paper presents a synthesis and extensions of earlier results.

MSC:

93E20	Optimal stochastic control
49J55	Existence of optimal solutions to problems involving randomness
60K20	Applications of Markov renewal processes (reliability, queueing networks, etc.)

Keywords:

stationary optimal policies; semi-Markov models; dynamic programming equations

References:

[1]	Bertsekas, D. P.; Shreve, S. E., Stochastic Optimal Control: The Discrete Time Case (1978), Academic Press: Academic Press New York · Zbl 0471.93002
[2]	Bhattacharya, R. N.; Lee, O., Asymptotics of a class of Markov processes which are not in general irreducible, Ann. Probab., 16, 1333-1347 (1988) · Zbl 0652.60028
[3]	Bhattacharya, R. N.; Majumdar, M., Stochastic models in mathematical economics: A review, (Proc. Golden Jubilee International Conference on Statistics: Applications and New Directions (1984), Indian Statistical Institute), 55-99
[4]	Bhattacharya, R. N.; Majumdar, M., Controlled semi-Markov models under long-run average rewards, J. Statist. Plann. Inference (1989), to appear · Zbl 0683.49003
[5]	Bhattacharya, R. N.; Waymire, E. C., Stochastic Processes with Applications (1989), Wiley: Wiley New York, to appear
[6]	Blackwell, David, Discounted dynamic programming, Ann. Math. Statist., 36, 226-235 (1965) · Zbl 0133.42805
[7]	Denardo, E. V., Contraction mappings in the theory underlying dynamic programming, SIAM Rev., 9, 165-177 (1967) · Zbl 0154.45101
[8]	Doob, J. L., Stochastic Processes (1953), Wiley: Wiley New York · Zbl 0053.26802
[9]	Furukawa, N., Markovian decision processes with compact action spaces, Ann. Math. Statist., 43, 1612-1622 (1972) · Zbl 0277.90083
[10]	Leland, H., Optimal growth in a stochastic environment, Rev. Econom. Stud., 41, 75-86 (1974) · Zbl 0281.90020
[11]	Levhari, D.; Srinivasan, T., Optimal savings under uncertainty, Rev. Econom. Stud., 36, 153-163 (1969)
[12]	Lippman, S. A., On dynamic programming with unbounded rewards, Management Sci., 21, 1225-1233 (1975) · Zbl 0309.90017
[13]	Maitra, A., Discounted dynamic programming on compact metric spaces, Sankhyā Ser. A., 30, 211-216 (1968) · Zbl 0187.17702
[14]	Mirman, L. J., One sector economic growth and uncertainty: A survey, (Dempster, M. A.H., Stochastic Programming (1980), Academic Press: Academic Press New York) · Zbl 0212.51704
[15]	Phelps, E., Accumulation of risky capital, Econometrica, 30, 729-743 (1962) · Zbl 0126.36402
[16]	Pliska, S. R., Controlled jump processes, Stochastic Process. Appl., 3, 259-282 (1975) · Zbl 0313.60055
[17]	Radner, R., Optimal growth in a linear logarithmic economy, Internat. Econom. Rev., 7, 1-35 (1966) · Zbl 0143.43104
[18]	Ross, S. M., Average cost semi-Markov decision processes, J. Appl. Probab., 7, 656-659 (1970) · Zbl 0204.51704
[19]	Rust, J., Optimal replacement of GMC bus engines: An empirical model of Harold Zurcher, Econometrica, 55, 999-1034 (1987) · Zbl 0624.90034
[20]	Rust, J., Maximum likelihood estimation of discrete control processes, SIAM J. Control and Optimization, 26, 1006-1024 (1988) · Zbl 0671.93060
[21]	Yushkevish, A., Controlled jump process, Soviet. Math. Dokl., 2, 351-355 (1975)