On the optimal control of a random walk with jumps and barriers.

*(English)*Zbl 1387.93183Summary: The modeling and optimal control of a class of random walks (RWs) is investigated in the framework of the Chapman-Kolmogorov (CK) and Fokker-Planck (FP) equations. This class of RWs includes jumps driven by a compound Poisson process and are subject to different barriers. A control mechanism is investigated that is included in the CK stochastic transition matrix and the purpose of the control is to track a desired discrete probability density function and attain a desired terminal density configuration. Existence and characterization of optimal controls are discussed. The proposed approach allows the derivation of a new FP model that accommodates the presence of the jumps and guarantees conservation of total probability in the case of reflecting barriers, which are modelled by appropriate operators. Results of numerical experiments are presented that successfully validate the proposed control framework.

##### MSC:

93E20 | Optimal stochastic control |

60G50 | Sums of independent random variables; random walks |

34K99 | Functional-differential equations (including equations with delayed, advanced or state-dependent argument) |

60J27 | Continuous-time Markov processes on discrete state spaces |

PDF
BibTeX
XML
Cite

\textit{T. Breitenbach} et al., Methodol. Comput. Appl. Probab. 20, No. 1, 435--462 (2018; Zbl 1387.93183)

Full Text:
DOI

##### References:

[1] | Amann H, Escher J (2008) Analysis II. Birkhäuser · Zbl 1146.26001 |

[2] | Annunziato M, Gottschalk H (2015) Calibration of Lévy processes using optimal control of Kolmogorov equations with periodic boundary conditions. arXiv:1506.08439 |

[3] | Arnold L (1974) Stochastic differential equations. Wiley, New York · Zbl 0278.60039 |

[4] | Bensoussan A (2011) Stochastic control by functional analysis methods, vol 11. Elsevier · Zbl 0474.93002 |

[5] | Beraldi, R, Random walk with long jumps for wireless ad hoc networks, Ad Hoc Netw, 7, 294-306, (2009) |

[6] | Beutler, FJ; Ross, KW, Optimal policies for controlled Markov chains with a constraint, J Math Anal Appl, 112, 236-252, (1985) · Zbl 0581.93067 |

[7] | Chang, JS; Cooper, G, A practical difference scheme for Fokker-Planck equations, J Comput Phys, 6, 1-16, (1970) · Zbl 0221.65153 |

[8] | Codling, EA; Plank, MJ; Benhamou, S, Random walk models in biology, J R Soc Interface, 5, 813-834, (2008) |

[9] | Cox DR, Miller HD (1965) The theory of stochastic processes. Wiley, New York · Zbl 0149.12902 |

[10] | Feller, W, Diffusion processes in one dimension, Trans Am Math Soc, 77, 1-31, (1954) · Zbl 0059.11601 |

[11] | Fleming WH, Soner HM (2006) Controlled Markov processes and viscosity solutions, vol 25. Springer Science & Business Media · Zbl 0059.11601 |

[12] | Friedman A (2006) Stochastic differential equations and applications. Dover Publications, Inc., New York · Zbl 1113.60003 |

[13] | Gaviraghi, B; Schindele, A; Annunziato, M; Borzì, A, On optimal sparse-control problems governed by jump-diffusion processes, Appl Math, 7, 1978, (2016) |

[14] | Hager, WW; Zhang, H, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J Optim, 16, 170-192, (2005) · Zbl 1093.90085 |

[15] | Luenberger DG (1969) Optimization by vector space methods. Wiley · Zbl 1093.90085 |

[16] | Milstein GN, Tretyakov MV (2013) Stochastic numerics for mathematical physics. Springer Science & Business Media |

[17] | Pugh CC (2015) Real mathematical analysis. Undergraduate texts in mathematics, 2nd edn. Springer, Cham · Zbl 1329.26003 |

[18] | Scalas, E, The application of continuous-time random walks in finance and economics, Physica A: Statistical Mechanics and its Applications, 362, 225-239, (2006) |

[19] | Yuste, SB; Abad, E; Lindenberg, K, Exploration and trapping of mortal random walkers, Phys Rev Lett, 110, 220603, (2013) |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.