×

zbMATH — the first resource for mathematics

Monotonicity of minimizers in optimization problems with applications to Markov control processes. (English) Zbl 1170.90513
Summary: Firstly, in this paper there is considered a certain class of possibly unbounded optimization problems on Euclidean spaces, for which conditions that permit to obtain monotone minimizers are given. Secondly, the theory developed in the first part of the paper is applied to Markov Control Processes (MCPs) on real spaces with possibly unbounded cost function, and with possibly noncompact control sets, considering both the discounted and the average cost as optimality criterion. In the context described, conditions to obtain monotone optimal policies are provided. For the conditions of MCPs presented in the article, several controlled models including, in particular, two inventory/production systems and the linear regulator problem are supplied.

MSC:
90C40 Markov and semi-Markov decision processes
93E20 Optimal stochastic control
PDF BibTeX XML Cite
Full Text: Link EuDML
References:
[1] Ash R. B.: Real Variables with Basic Metric Space Topology. IEEE Press, New York 1993 · Zbl 0920.26002
[2] Cruz-Suárez D., Montes-de-Oca, R., Salem-Silva F.: Conditions for the uniqueness of optimal policies of discounted Markov decision processes. Math. Methods Oper. Res. 60 (2004), 415-436 · Zbl 1104.90053 · doi:10.1007/s001860400372
[3] Cruz-Suárez D., Montes-de-Oca, R., Salem-Silva F.: Pointwise approximations of discounted Markov decision processes to optimal policies. Internat. J. Pure Appl. Math. 28 (2006), 265-281 · Zbl 1131.90068
[4] Fu M. C., Marcus S. I., Wang, I-J: Monotone optimal policies for a transient queueing staffing problem. Oper. Res. 48 (2000), 327-331
[5] Gallish E.: On monotone optimal policies in a queueing model of M/G/1 type with controllable service time distribution. Adv. in Appl. Probab. 11 (1979), 870-887 · Zbl 0434.60095 · doi:10.2307/1426864
[6] Hernández-Lerma O., Lasserre J. B.: Discrete-Time Markov Control Processes. Springer-Verlag, New York 1996 · Zbl 0928.93002
[7] Heyman D. P., Sobel M. J.: Stochastic Models in Operations Research, Vol. II. Stochastic Optimization. McGraw-Hill, New York 1984 · Zbl 1072.90001
[8] Hinderer K., Stieglitz M.: Increasing and Lipschitz continuous minimizers in one-dimensional linear-convex systems without constraints: The continuous and the discrete case. Math. Methods Oper. Res. 44 (1996), 189-204 · Zbl 0860.90126
[9] Kalin D.: A note on ‘monotone optimal policies for Markov decision processes’. Math. Programming 15 (1978), 220-222 · Zbl 0387.90106 · doi:10.1007/BF01609021
[10] Mendelssohn R., Sobel M.: Capital accumulation and the optimization of renewable resource models. J. Econom. Theory 23 (1980), 243-260 · Zbl 0472.90015 · doi:10.1016/0022-0531(80)90009-5
[11] Pittenger A. O.: Monotonicity in a Markov decision process. Math. Oper. Res. 13 (1988), 65-73 · Zbl 0646.90088 · doi:10.1287/moor.13.1.65
[12] Porteus E. L.: Foundations of Stochastic Inventory Theory. Stanford University Press, Stanford, Calif. 2002
[13] Puterman M. L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York 1994 · Zbl 1184.90170
[14] Rieder U.: Measurable selection theorems for optimization problems. Manuscripta Math. 24 (1978), 115-131 · Zbl 0385.28005 · doi:10.1007/BF01168566 · eudml:154537
[15] Ross S. M.: Introduction to Stochastic Dynamic Programming. Academic Press, San Diego 1983 · Zbl 0567.90065
[16] Serfozo R. F.: Monotone optimal policies for Markov decision processes. Math. Programming Stud. 6 (1976), 202-215 · Zbl 0368.60080
[17] Stidham, Sh., Weber R. R.: Monotonic and insensitive optimal policies for control of queues with undiscounted costs. Oper. Res. 37 (1989), 611-625 · Zbl 0674.90029 · doi:10.1287/opre.37.4.611
[18] Stromberg K. R.: An Introduction to Classical Real Analysis. Wadsworth International Group, Belmont 1981 · Zbl 0454.26001
[19] Sundaram R. K.: A First Course in Optimization Theory. Cambridge University Press, Cambridge 1996 · Zbl 0885.90106 · doi:10.1017/CBO9780511804526
[20] Topkis D. M.: Minimizing a submodular function on a lattice. Oper. Res. 26 (1978), 305-321 · Zbl 0379.90089 · doi:10.1287/opre.26.2.305
[21] Topkis D. M.: Supermodularity and Complementarity. Princeton University Press, Princeton, N. J. 1988
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.