×

zbMATH — the first resource for mathematics

An approximation scheme for the optimal control of diffusion processes. (English) Zbl 0822.65044
A numerical approximation scheme for the infinite horizon discounted optimal control problem for a stochastic diffusion process is presented. The scheme is based on a discrete version of the dynamic programming principle and converges to the weak (viscosity) solution of the Hamilton- Jacobi-Bellman equation. The convergence is valid also when the diffusion is degenerate.
Since a diffusion leaves any bounded domain with probability 1, for the problem in \(\mathbb{R}^ n\) a truncation technique is proposed to restrict the problem to an arbitrary bounded domain. Explicit estimates of the error due to the truncation technique are proved. The method provides also approximate feedback controls at any point of the grid without extra computations. Numerical tests are given.
Reviewer: V.Arnăutu (Iaşi)

MSC:
65K10 Numerical optimization and variational techniques
49L20 Dynamic programming in optimal control and differential games
49L25 Viscosity solutions to Hamilton-Jacobi equations in optimal control and differential games
49J55 Existence of optimal solutions to problems involving randomness
60J60 Diffusion processes
PDF BibTeX XML Cite
Full Text: DOI EuDML
References:
[1] M. AKIAN, 1990, Analyse de l’algorithme multigrille FMGH de résolution d’équations de Hamilton-Jacobi-Bellman, in A. Bensoussan, J.-L Lions (Eds.), Analysis and Optimization of Systems, Lecture Notes in Control and Information Science, n^\circ 144, Springer-Verlag, 113-122. Zbl0712.93069 MR1070728 · Zbl 0712.93069
[2] G. BARLES, P. E. SOUGANIDIS, 1991, Convergence of approximation schemes for fully non linear second order equations, Asymptotic Analysis, 4, 271-282. Zbl0729.65077 MR1115933 · Zbl 0729.65077
[3] M.-C. BANCORA-IMBERT, P.-L. CHOW, J.-L. MENALDI, 1989, On the numerical approximation of an optimal correction problem, SIAM J. Sci Statist. Comput., 9, 970-991. Zbl0661.65150 MR963850 · Zbl 0661.65150 · doi:10.1137/0909068
[4] D. P. BERTSEKAS, S. SHREVE, 1978, Stochastic Optimal Control: the discrete time case, Academic Press, New York. Zbl0471.93002 MR511544 · Zbl 0471.93002
[5] I. CAPUZZO-DOLCETTA, , 1983, On a discrete approximation of the Hamilton-Jacobi equation of Dynamic Programming, Appl. Math. Optim., 10, 367-377. Zbl0582.49019 MR713483 · Zbl 0582.49019 · doi:10.1007/BF01448394
[6] [CDF] I. CAPUZZO-DOLCETTA, M. FALCONE, 1989, Discrete dynamic programming and viscosity solution of the Bellman equation, Annales de l’Institut H. Poincaré-Analyse non linéaire, 6, 161-184. Zbl0674.49028 MR1019113 · Zbl 0674.49028 · numdam:AIHPC_1989__S6__161_0 · eudml:78193
[7] F. CAMILLI, M. FALCONE, forthcoming.
[8] I. CAPUZZO-DOLCETTA, H. ISHII, 1984, Approximate solutions of the Bellman equation of Deterministic Control Theory, Appl. Math. Optim., 11, 161-181. Zbl0553.49024 MR743925 · Zbl 0553.49024 · doi:10.1007/BF01442176
[9] M. CRANDALL, H. ISHII, P.-L. LIONS, 1991, A user guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc., 27, 1-67. Zbl0755.35015 MR1118699 · Zbl 0755.35015 · doi:10.1090/S0273-0979-1992-00266-5
[10] M. FALCONE, 1987, 1991, A numerical approach to infinite horizon problem,Appl. Math. Optim., 15, 1-13 and 23, 213-214. Zbl0715.49023 MR866164 · Zbl 0715.49023 · doi:10.1007/BF01442644
[11] M. FALCONE, 1985, Numerical solution of optimal control problems, Proceedings of the International Symposium on Numerical Analysis, Madrid. · Zbl 0599.92018
[12] M. FALCONE, R. FERRETTI, 1994, Discrete-time high-order schemes for viscosity solutions of Hamilton-Jacobi-Bellman equations, Numerische Mathematik, 67,315-344. Zbl0791.65046 MR1269500 · Zbl 0791.65046 · doi:10.1007/s002110050031
[13] B. G. FITZPATRICK, W. H. FLEMING, Numerical Methods for an optimal investment/consumption model, to appear on Math. Operation Research. Zbl0744.90003 MR1135050 · Zbl 0744.90003 · doi:10.1287/moor.16.4.823
[14] W. H. FLEMING, M. H. SONER, 1993, Controlled Markov processes and viscosity solutions, Springer-Verlag, New York. Zbl0773.60070 MR1199811 · Zbl 0773.60070
[15] R. GONZALES, E. ROFMAN, 1985, On determmistic control problems : an approximation procedure for the optimal cost(part I and II), SIAM J. Control and Optimization, 23, 242-285. Zbl0563.49025 · Zbl 0563.49025 · doi:10.1137/0323019
[16] [H] R. HOPPE, 1986, Multi-grid methods for Hamilton-Jacobi-Bellman equations, Numerische Mathematik, 49, 235-254. Zbl0577.65088 MR848524 · Zbl 0577.65088 · doi:10.1007/BF01389627 · eudml:133110
[17] H. ISHII, P.-L. LIONS, 1990, Viscosity solutions of fully nonlinear second-order elliptic partial differential equations, J. Diff. Eq., 83, 26-78. Zbl0708.35031 MR1031377 · Zbl 0708.35031 · doi:10.1016/0022-0396(90)90068-Z
[18] H. J. KUSHNER, 1977, Probability methods for approximations in stochastic control and for elliptic equations, Academic Press, New York. Zbl0547.93076 MR469468 · Zbl 0547.93076
[19] H. J. KUSHNER, P. DUPUIS, 1992, Numerical methods for stochastic control problems in continuous time, Springer Verlag, New York. Zbl0754.65068 MR1217486 · Zbl 0754.65068
[20] P.-L. LIONS, 1983, Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations part 1 the dynamic programming principle and applications, Comm. Partial Diff. Equations, 8, 1101-1174. Zbl0716.49022 MR709164 · Zbl 0716.49022 · doi:10.1080/03605308308820297
[21] P.-L. LIONS, 1983, Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations part 2 : Viscosity solutions and uniqueness, Comm. Partial Diff. Equations, 8, 1229-1270. Zbl0716.49023 MR709162 · Zbl 0716.49023 · doi:10.1080/03605308308820301
[22] [LM] P.-L. LIONS, B. MERCIER, Approximation numérique des équations de Hamilton-Jacobi-Bellman, RAIRO Anal. Numer., 14, 369-393. Zbl0469.65041 MR596541 · Zbl 0469.65041 · eudml:193367
[23] J.-L. MENALDI, 1989, Some estimates for finite difference approximations, SIAM J. Control Optim., 27, 579-607. Zbl0684.93088 MR993288 · Zbl 0684.93088 · doi:10.1137/0327031
[24] J.-P. QUADRAT, 1980, Existence de solution et algorithme de résolutions numériques de problèmes stochastiques dégénérées ou non, SIAM J. Control and Optimization, 18, 199-226. Zbl0439.93057 MR560048 · Zbl 0439.93057 · doi:10.1137/0318015
[25] P.-E. SOUGANIDIS, 1985, Approximation schemes for viscosity solutions of Hamilton-Jacobi equations, J. Diff. Eq., 57, 1-43. Zbl0536.70020 MR803085 · Zbl 0536.70020 · doi:10.1016/0022-0396(85)90136-6
[26] M. SUN, 1993, Domain decomposition algorithms for solvign Hamilton-Jacobi-Bellman equations, Numerical Func. Anal. and Optim., 14, 145-166. Zbl0810.65065 MR1210467 · Zbl 0810.65065 · doi:10.1080/01630569308816513
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.