Stability estimating in optimal stopping problem. (English) Zbl 1154.60326

Summary: We consider the optimal stopping problem for a discrete-time Markov process on a Borel state space \(X\). It is supposed that an unknown transition probability \(p(\cdot |x)\), \(x\in X\), is approximated by the transition probability \(\widetilde{p}(\cdot |x)\), \(x\in X\), and the stopping rule \(\widetilde{\tau}_*\), optimal for \(\widetilde{p}\), is applied to the process governed by \(p\). We found an upper bound for the difference between the total expected cost, resulting when applying \(\widetilde{\tau}_*\), and the minimal total expected cost. The bound given is a constant times \(\sup_{x\in X}\|p(\cdot |x)-\widetilde{p}(\cdot |x)\|\), where \(\|\cdot\|\) is the total variation norm.


60G40 Stopping times; optimal stopping problems; gambling theory
Full Text: EuDML Link


[1] Allart P.: Optimal stopping rules for correlated random walks with a discount. J. Appl. Prob. 41 (2004), 483-496 · Zbl 1071.60026
[2] Bertsekas D. P.: Dynamic Programming: Deterministic and Stochastic Models. Prentice Hall, Englewood Cliffs, N. J. 1987 · Zbl 0649.93001
[3] Bertsekas D. P., Shreve S. E.: Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York 1979 · Zbl 0633.93001
[4] Dijk N. M. Van: Perturbation theory for unbounded Markov reward process with applications to queueing systems. Adv. in Appl. Probab. 20 (1988), 99-111 · Zbl 0642.60099
[5] Dijk N. M. Van, Sladký K.: Error bounds for nonnegative dynamic models. J. Optim. Theory Appl. 101 (1999), 449-474 · Zbl 0946.90106
[6] Dynkin E. B., Yushkevich A. A.: Controlled Markov Process. Springer-Verlag, New York 1979
[7] Favero G., Runggaldier W. J.: A robustness results for stochastic control. Systems Control Lett. 46 (2002), 91-97 · Zbl 0994.93008
[8] Gordienko E. I.: An estimate of the stability of optimal control of certain stochastic and deterministic systems. J. Soviet Math. 59 (1992), 891-899. ( · Zbl 1267.49026
[9] Gordienko E. I., Salem F. S.: Robustness inequality for Markov control process with unbounded costs. Systems Control Lett. 33 (1998), 125-130 · Zbl 0902.93068
[10] Gordienko E. I., Yushkevich A. A.: Stability estimates in the problem of average optimal switching of a Markov chain. Math. Methods Oper. Res. 57 (2003), 345-365 · Zbl 1116.90401
[11] Gordienko E. I., Lemus-Rodríguez E., Montes-de-Oca R.: Discounted cost optimality problem: stability with respect to weak metrics. In press in: Math. Methhods Oper. Res. (2008) · Zbl 1166.60041
[12] Hernández-Lerma O., Lassere J. B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, N.Y. 1996
[13] Jensen U.: An optimal stopping problem in risk theory. Scand. Actuarial J.2 (1997), 149-159 · Zbl 0888.62104
[14] Meyn S. P., Tweedie R. L.: Markov Chains and Stochastic Stability. Springer-Verlag, London 1993 · Zbl 1165.60001
[15] Montes-de-Oca R., Salem-Silva F.: Estimates for perturbations of an average Markov decision process with a minimal state and upper bounded by stochastically ordered Markov chains. Kybernetika 41 (2005), 757-772 · Zbl 1249.90313
[16] Montes-de-Oca R., Sakhanenko, A., Salem-Silva F.: Estimate for perturbations of general discounted Markov control chains. Appl. Math. 30 (2003), 287-304 · Zbl 1055.90086
[17] Muciek B. K.: Optimal stopping of a risk process: model with interest rates. J. Appl. Prob. 39 (2002), 261-270 · Zbl 1011.62111
[18] Müller A.: How does the value function of a Markov decision process depend on the transition probabilities? Math. Oper. Res. 22 (1997), 872-885 · Zbl 0892.90178
[19] Schäl M.: Conditions for optimality in dynamic programming and for the limit of \(n\)-stage optimal policies to be optimal. Z. Wahrsch. verw. Gebiete 32 (1975), 179-196 · Zbl 0316.90080
[20] Shiryaev A. N.: Optimal Stopping Rules. Springer-Verlag, New York 1978 · Zbl 1138.60008
[21] Shiryaev A. N.: Essential of Stochastic Finance. Facts, Models, Theory. World Scientific Publishing Co., Inc., River Edge, N.J. 1999 · Zbl 0926.62100
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.