Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion. (English) Zbl 1190.93105

Summary: The paper deals with a class of discrete-time stochastic control processes under a discounted optimality criterion with random discount rate, and possibly unbounded costs. The state process \(x_t\) and the discount process \(\alpha_t\) evolve according to the coupled difference equations \(x_{t+1}= F(x_t\alpha_t,a_t,\xi_t)\), \(\alpha_{t+1}= G(\alpha_t,\eta_t)\) where the state and discount disturbance processes \(\xi_t\) and \(\eta_t\) are sequences of i.i.d. random variables with densities \(\rho^\xi\) and \(\rho^\eta\) respectively. The main objective is to introduce approximation algorithms of the optimal cost function that lead up to construction of optimal or nearly optimal policies in the cases when the densities \(\rho^\xi\) and \(\rho^\eta\) are either known or unknown. In the latter case, we combine suitable estimation methods with control procedures to construct an asymptotically discounted optimal policy.


93E20 Optimal stochastic control
90C40 Markov and semi-Markov decision processes
93E10 Estimation and detection in stochastic control theory
93C55 Discrete-time control/observation systems
Full Text: Link


[1] H. Berument, Z. Kilinc, and U. Ozlale: The effects of different inflation risk premiums on interest rate spreads. Physica A 333 (2004), 317-324.
[2] L. Devroye and L. Györfi: Nonparametric Density Estimation the \(L_{1}\) View. Wiley, New York 1985. · Zbl 0546.62015
[3] E. B. Dynkin and A. A. Yushkevich: Controlled Markov Processes. Springer-Verlag, New York 1979.
[4] A. Gil and A. Luis: Modelling the U. S. interest rate in terms of I(d) statistical model. Quart. Rev. Economics and Finance 44 (2004), 475-486.
[5] R. Hasminskii and I. Ibragimov: On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Statist. 18 (1990), 999-1010. · Zbl 0705.62039
[6] J. González-Hernández, R. R. López-Martínez, and R. Pérez-Hernández: Markov control processes with randomized discounted cost in Borel space. Math. Meth. Oper. Res. 65 (2007), 27-44. · Zbl 1126.90075
[7] S. Haberman and J. Sung: Optimal pension funding dynamics over infinite control horizon when stochastic rates of return are stationary. Insurance Math. Econom. 36 (2005), 103-116. · Zbl 1111.91023
[8] O. Hernández-Lerma: Adaptive Markov Control Processes. Springer-Verlag, New York 1989. · Zbl 0698.90053
[9] O. Hernández-Lerma and J. B. Lasserre: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, New York 1996.
[10] O. Hernández-Lerma and J. B. Lasserre: Further Topics on Discrete-Time Markov Control Processes. Springer-Verlag, New York 1999. · Zbl 0928.93002
[11] O. Hernández-Lerma and W. Runggaldier: Monotone approximations for convex stochastic control problems. J. Math. Syst., Estimation, and Control 4 (1994), 99-140. · Zbl 0812.93078
[12] P. Lee and D. B. Rosenfield: When to refinance a mortgage: a dynamic programming approach. European J. Oper. Res. 166 (2005), 266-277. · Zbl 1066.91022
[13] R. G. Newell and W. A. Pizer: Discounting the distant future: how much do uncertain rates increase valuation? J. Environmental Economic and Management 46 (2003), 52-71. · Zbl 1041.91502
[14] B. Sack and V. Wieland: Interest-rate smooothing and optimal monetary policy: A review of recent empirical evidence. J. Econom. Business 52 (2000), 205-228.
[15] M. Schäl: Conditions for optimality and for the limit of n-stage optimal policies to be optimal. Z. Wahrsch. Verw. Gerb. 32 (1975), 179-196. · Zbl 0316.90080
[16] M. Schäl: Estimation and control in discounted stochastic dynamic programming. Stochastics 20 (1987), 51-71. · Zbl 0621.90092
[17] N. L. Stockey and R. E. Lucas, Jr.: Recursive Methods in Economic Dynamics. Harvard University Press, Cambridge, MA 1989.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.