Stability estimating in optimal stopping problem. (English) Zbl 1154.60326

Zaitseva, Elena

Kybernetika 44, No. 3, 400-415 (2008).

Summary: We consider the optimal stopping problem for a discrete-time Markov process on a Borel state space \(X\). It is supposed that an unknown transition probability \(p(\cdot |x)\), \(x\in X\), is approximated by the transition probability \(\widetilde{p}(\cdot |x)\), \(x\in X\), and the stopping rule \(\widetilde{\tau}_*\), optimal for \(\widetilde{p}\), is applied to the process governed by \(p\). We found an upper bound for the difference between the total expected cost, resulting when applying \(\widetilde{\tau}_*\), and the minimal total expected cost. The bound given is a constant times \(\sup_{x\in X}\|p(\cdot |x)-\widetilde{p}(\cdot |x)\|\), where \(\|\cdot\|\) is the total variation norm.

MSC:

60G40

Stopping times; optimal stopping problems; gambling theory

Keywords:

discrete-time Markov process; optimal stopping rule; stability index; total variation metric; contractive operator; optimal asset selling

References:

[1]	Allart P.: Optimal stopping rules for correlated random walks with a discount. J. Appl. Prob. 41 (2004), 483-496 · Zbl 1071.60026
[2]	Bertsekas D. P.: Dynamic Programming: Deterministic and Stochastic Models. Prentice Hall, Englewood Cliffs, N. J. 1987 · Zbl 0649.93001
[3]	Bertsekas D. P., Shreve S. E.: Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York 1979 · Zbl 0633.93001
[4]	Dijk N. M. Van: Perturbation theory for unbounded Markov reward process with applications to queueing systems. Adv. in Appl. Probab. 20 (1988), 99-111 · Zbl 0642.60099
[5]	Dijk N. M. Van, Sladký K.: Error bounds for nonnegative dynamic models. J. Optim. Theory Appl. 101 (1999), 449-474 · Zbl 0946.90106
[6]	Dynkin E. B., Yushkevich A. A.: Controlled Markov Process. Springer-Verlag, New York 1979
[7]	Favero G., Runggaldier W. J.: A robustness results for stochastic control. Systems Control Lett. 46 (2002), 91-97 · Zbl 0994.93008
[8]	Gordienko E. I.: An estimate of the stability of optimal control of certain stochastic and deterministic systems. J. Soviet Math. 59 (1992), 891-899. ( · Zbl 1267.49026
[9]	Gordienko E. I., Salem F. S.: Robustness inequality for Markov control process with unbounded costs. Systems Control Lett. 33 (1998), 125-130 · Zbl 0902.93068
[10]	Gordienko E. I., Yushkevich A. A.: Stability estimates in the problem of average optimal switching of a Markov chain. Math. Methods Oper. Res. 57 (2003), 345-365 · Zbl 1116.90401
[11]	Gordienko E. I., Lemus-Rodríguez E., Montes-de-Oca R.: Discounted cost optimality problem: stability with respect to weak metrics. In press in: Math. Methhods Oper. Res. (2008) · Zbl 1166.60041
[12]	Hernández-Lerma O., Lassere J. B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, N.Y. 1996
[13]	Jensen U.: An optimal stopping problem in risk theory. Scand. Actuarial J.2 (1997), 149-159 · Zbl 0888.62104
[14]	Meyn S. P., Tweedie R. L.: Markov Chains and Stochastic Stability. Springer-Verlag, London 1993 · Zbl 1165.60001
[15]	Montes-de-Oca R., Salem-Silva F.: Estimates for perturbations of an average Markov decision process with a minimal state and upper bounded by stochastically ordered Markov chains. Kybernetika 41 (2005), 757-772 · Zbl 1249.90313
[16]	Montes-de-Oca R., Sakhanenko, A., Salem-Silva F.: Estimate for perturbations of general discounted Markov control chains. Appl. Math. 30 (2003), 287-304 · Zbl 1055.90086
[17]	Muciek B. K.: Optimal stopping of a risk process: model with interest rates. J. Appl. Prob. 39 (2002), 261-270 · Zbl 1011.62111
[18]	Müller A.: How does the value function of a Markov decision process depend on the transition probabilities? Math. Oper. Res. 22 (1997), 872-885 · Zbl 0892.90178
[19]	Schäl M.: Conditions for optimality in dynamic programming and for the limit of \(n\)-stage optimal policies to be optimal. Z. Wahrsch. verw. Gebiete 32 (1975), 179-196 · Zbl 0316.90080
[20]	Shiryaev A. N.: Optimal Stopping Rules. Springer-Verlag, New York 1978 · Zbl 1138.60008
[21]	Shiryaev A. N.: Essential of Stochastic Finance. Facts, Models, Theory. World Scientific Publishing Co., Inc., River Edge, N.J. 1999 · Zbl 0926.62100