×

zbMATH — the first resource for mathematics

An envelope theorem and some applications to discounted Markov decision processes. (English) Zbl 1149.90171
This paper considers an Envelope Theorem for optimization problems on the Euclidean space under two different class conditions: One is a concavity assumptions, the other are differentiability conditions in the transition law, in the reward function, and the noise of the system. The criterion in this paper is expected total discounted reward. Some interesting examples of economic models are also presented.

MSC:
90C40 Markov and semi-Markov decision processes
93E20 Optimal stochastic control
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Aliprantis CD, Burkinshaw O (1998) Principles of real analysis. Academic, New York · Zbl 1006.28001
[2] Amir R (1997) A new look at optimal growth under uncertainty. J Econ Dynam Control 22:67–86 · Zbl 0897.90050
[3] Aoki M (1989) Optimization of stochastic systems, 2nd edn. Academic, New York · Zbl 0699.93098
[4] Araujo A, Scheinkman JA (1977) Smoothness, comparative dynamics, and the turnpike property. Econometrica 45:601–620 · Zbl 0381.90107
[5] Arkin VI, Evstigneev IV (1987) Stochastic models of control and economic dynamic. Academic, New York
[6] Aubin JP (1982) Mathematical methods of game and economic theory. North Holland, Amsterdam
[7] Benveniste LM, Scheinkman JA (1979) On the differentiability of the value function in dynamic models of economics. Econometrica 47:727–732 · Zbl 0435.90031
[8] Bertsekas DP (1987) Dynamic programming: deterministic and stochastic models. Prentice-Hall, Englewood Cliffs · Zbl 0649.93001
[9] Billingley P (1986) Probability and measure. Wiley, New York
[10] Blume L, Easley D, O’Hara M (1982) Characterization of optimal plans for stochastic dynamic programs. J Econ Theory 28:221–234 · Zbl 0509.90021
[11] Brock WA, Mirman LJ (1972) Optimal economic growth and uncertainty: the discounted case. J Econ Theory 4:479–513
[12] Cruz-Suárez D, Montes-de-Oca R, Salem-Silva F (2004) Conditions for the uniqueness of optimal policies of discounted Markov decision processes. Math Methods Oper Res 60:415–436 · Zbl 1104.90053
[13] Cruz-Suárez H, Montes-de-Oca R (2006) Discounted Markov control processes induced by deterministic systems. To appear in Kybernetika (Prague) · Zbl 1249.90312
[14] De la Fuente A (2000) Mathematical methods and models for economists. Cambridge University Press, Cambridge · Zbl 0943.91001
[15] Duffie D (1988) Security markets. Academic, New York
[16] Duffie D (2000) Dynamic asset pricing theory. Princeton University Press, Princeton · Zbl 1140.91041
[17] Dynkin E, Yushkevich A (1979) Controlled Markov processes. Springer, New York · Zbl 0073.34801
[18] Harris M (1987) Dynamic economic analysis. Oxford University Press, New York
[19] Hernández-Lerma O, Lasserre JB (1993) Value iteration and rolling plans for Markov control processes with unbounded rewards. J Math Anal Appl 177:38–55 · Zbl 0781.90093
[20] Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria. Springer, New York
[21] Judd KL (1999) Numerical methods in economics. The MIT Press, Cambridge
[22] Kraus M (2002) A generalized envelope theorem with an application to congestion-prone facilities. Econ Bull 3:1–4
[23] Levhari D, Srinivasan TN (1969) Optimal savings under uncertainty. Rev Econ Stud 36:153–164
[24] Marsden JE (1974) Elementary classical analysis. WH Freeman, San Francisco · Zbl 0285.26005
[25] Milgrom P, Segal I (2002) Envelope theorems for arbitrary choice sets. Econometrica 70:583–601 · Zbl 1103.90400
[26] Mirman LJ (1979) Dynamic models of fishing: a heuristic approach. In Pan-Tai L, Sutinen JG (eds) Control theory in mathematical economics. Marcel Dekker Inc, New York, pp 39–73
[27] Santos MS (1994) Smooth dynamics and computation in models of economic growth. J Econ Dynam Control 18:879–895 · Zbl 0875.62607
[28] Santos MS (1999) Numerical solution of dynamic economic models. In: Taylor JB, Woodford M (eds) Handbook of macroeconomic, vol I. North Holland, Amsterdam, pp 311–386
[29] Stokey NL, Lucas RE (1989) Recursive methods in economic dynamics. Harvard University Press, Cambridge
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.