Diffusion approximations for controlled weakly interacting large finite state systems with simultaneous jumps. (English) Zbl 1391.60232

Summary: We consider a rate control problem for an \(N\)-particle weakly interacting finite state Markov process. The process models the state evolution of a large collection of particles and allows for multiple particles to change state simultaneously. Such models have been proposed for large communication systems (e.g., ad hoc wireless networks) but are also suitable for other settings such as chemical-reaction networks. An associated diffusion control problem is presented and we show that the value function of the \(N\)-particle controlled system converges to the value function of the limit diffusion control problem as \(N\to\infty\). The diffusion coefficient in the limit model is typically degenerate; however, under suitable conditions there is an equivalent formulation in terms of a controlled diffusion with a uniformly nondegenerate diffusion coefficient. Using this equivalence, we show that near optimal continuous feedback controls exist for the diffusion control problem. We then construct near asymptotically optimal control policies for the \(N\)-particle system based on such continuous feedback controls. Results from some numerical experiments are presented.


60K35 Interacting random processes; statistical mechanics type models; percolation theory
60H30 Applications of stochastic analysis (to PDEs, etc.)
93E20 Optimal stochastic control
60J28 Applications of continuous-time Markov processes on discrete state spaces
60J70 Applications of Brownian motions and diffusion theory (population genetics, absorption problems, etc.)
60K25 Queueing theory (aspects of probability theory)
91B70 Stochastic models in economics
Full Text: DOI arXiv


[1] Antunes, N., Fricker, C., Robert, P. and Tibi, D. (2008). Stochastic networks with multiple stable points. Ann. Probab.36 255-278. · Zbl 1130.60086
[2] Atar, R. (2005). A diffusion model of scheduling control in queueing systems with many servers. Ann. Appl. Probab.15 820-852. · Zbl 1084.60053
[3] Bell, S. L. and Williams, R. J. (2001). Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: Asymptotic optimality of a threshold policy. Ann. Appl. Probab.11 608-649. · Zbl 1015.60080
[4] Bensoussan, A. (1982). Stochastic Control by Functional Analysis Methods. Studies in Mathematics and Its Applications 11. North-Holland, Amsterdam. · Zbl 0474.93002
[5] Billingsley, P. (1999). Convergence of Probability Measures, 2nd ed. Wiley, New York. · Zbl 0944.60003
[6] Borkar, V. S. (1989). Optimal Control of Diffusion Processes. Pitman Research Notes in Mathematics Series 203. Longman Scientific & Technical, Harlow. · Zbl 0669.93065
[7] Budhiraja, A. and Ghosh, A. P. (2012). Controlled stochastic networks in heavy traffic: Convergence of value functions. Ann. Appl. Probab.22 734-791. · Zbl 1244.60091
[8] Budhiraja, A., Ghosh, A. P. and Lee, C. (2011). Ergodic rate control problem for single class queueing networks. SIAM J. Control Optim.49 1570-1606. · Zbl 1226.93138
[9] Chen, Z. and Huan, Z. (1997). On the continuity of the \(m\)th root of a continuous nonnegative definite matrix-valued function. J. Math. Anal. Appl.209 60-66. · Zbl 0880.15018
[10] Dai, J. G. and Lin, W. (2008). Asymptotic optimality of maximum pressure policies in stochastic processing networks. Ann. Appl. Probab.18 2239-2299. · Zbl 1175.90083
[11] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York. · Zbl 0592.60049
[12] Fleming, W. H. and Rishel, R. W. (1975). Deterministic and Stochastic Optimal Control. Applications of Mathematics 1. Springer, Berlin. · Zbl 0323.49001
[13] Harrison, J. M. (1988). Brownian models of queueing networks with heterogeneous customer populations. In Stochastic Differential Systems, Stochastic Control Theory and Applications (Minneapolis, Minn., 1986). IMA Vol. Math. Appl.10 147-186. Springer, New York.
[14] Joffe, A. and MĂ©tivier, M. (1986). Weak convergence of sequences of semimartingales with applications to multitype branching processes. Adv. in Appl. Probab.18 20-65. · Zbl 0595.60008
[15] Krylov, N. V. (2009). Controlled Diffusion Processes. Stochastic Modelling and Applied Probability 14. Springer, Berlin.
[16] Kurtz, T. G. (1970). Solutions of ordinary differential equations as limits of pure jump Markov processes. J. Appl. Probab.7 49-58. · Zbl 0191.47301
[17] Kurtz, T. G. (1971). Limit theorems for sequences of jump Markov processes approximating ordinary differential processes. J. Appl. Probab.8 344-356. · Zbl 0219.60060
[18] Kurtz, T. G. and Protter, P. (1991). Weak limit theorems for stochastic integrals and stochastic differential equations. Ann. Probab.19 1035-1070. · Zbl 0742.60053
[19] Kushner, H. J. (2001). Heavy Traffic Analysis of Controlled Queueing and Communication Networks. Applications of Mathematics (New York): Stochastic Modelling and Applied Probability 47. Springer, New York. · Zbl 0988.90004
[20] Protter, P. E. (2005). Stochastic Integration and Differential Equations, 2nd ed. Stochastic Modelling and Applied Probability 21. Springer, Berlin.
[21] Rudin, W. (1986). Real and Complex Analysis, 3rd ed. McGraw-Hill, New York. · Zbl 0925.00005
[22] Stroock, D. W. and Varadhan, S. R. S. (2006). Multidimensional Diffusion Processes. Springer, Berlin. · Zbl 1103.60005
[23] Whitt, W.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.