×

zbMATH — the first resource for mathematics

Stochastic approximation of quasi-stationary distributions on compact spaces and applications. (English) Zbl 06974754
Summary: As a continuation of a recent paper, dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact metric space killed in finite time. The idea is to run the process until extinction and then to bring it back to life at a position randomly chosen according to the (possibly weighted) empirical occupation measure of its past positions. General conditions are given ensuring the convergence of this measure to the quasi-stationary distribution of the chain. We then apply this method to the numerical approximation of the quasi-stationary distribution of a diffusion process killed on the boundary of a compact set. Finally, the sharpness of the assumptions is illustrated through the study of the algorithm in a nonirreducible setting.

MSC:
65C20 Probabilistic models, generic numerical methods in probability and statistics
60B12 Limit theorems for vector-valued random variables (infinite-dimensional case)
60J10 Markov chains (discrete-time Markov processes on discrete state spaces)
34F05 Ordinary differential equations and systems with randomness
60J20 Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.)
60J60 Diffusion processes
PDF BibTeX Cite
Full Text: DOI Euclid
References:
[1] Aldous, D., Flannery, B. and Palacios, J. (1988). Two applications of urn processes: The fringe analysis of search trees and the simulation of quasi-stationary distributions of Markov chains. Probab. Engrg. Inform. Sci.2 293–307. · Zbl 1134.68592
[2] Bansaye, V., Méléard, S. and Richard, M. (2016). Speed of coming down from infinity for birth-and-death processes. Adv. in Appl. Probab.48 1183–1210. · Zbl 1358.60087
[3] Bass, R. F. (1998). Diffusions and Elliptic Operators. Springer, New York. · Zbl 0914.60009
[4] Ben-Ari, I. and Pinsky, R. G. (2007). Spectral analysis of a family of second-order elliptic operators with nonlocal boundary condition indexed by a probability measure. J. Funct. Anal.251 122–140. · Zbl 1137.35045
[5] Benaïm, M. (1999). Dynamics of stochastic approximation algorithms. In Séminaire de Probabilités, XXXIII. Lecture Notes in Math.1709 1–68. Springer, Berlin.
[6] Benaïm, M. and Cloez, B. (2015). A stochastic approximation approach to quasi-stationary distributions on finite spaces. Electron. Commun. Probab.20 no. 37. · Zbl 1321.65009
[7] Benaïm, M. and Hirsch, M. W. (1996). Asymptotic pseudotrajectories and chain recurrent flows, with applications. J. Dynam. Differential Equations8 141–176. · Zbl 0878.58053
[8] Benaïm, M., Ledoux, M. and Raimond, O. (2002). Self-interacting diffusions. Probab. Theory Related Fields122 1–41. · Zbl 1042.60060
[9] Berglund, N. and Landon, D. (2012). Mixed-mode oscillations and interspike interval statistics in the stochastic FitzHugh–Nagumo model. Nonlinearity25 2303–2335. · Zbl 1248.60059
[10] Bieniek, M., Burdzy, K. and Finch, S. (2012). Non-extinction of a Fleming–Viot particle model. Probab. Theory Related Fields153 293–332. · Zbl 1253.60089
[11] Bieniek, M., Burdzy, K. and Pal, S. (2012). Extinction of Fleming–Viot-type particle systems with strong drift. Electron. J. Probab.17 no. 11. · Zbl 1258.60031
[12] Blanchet, J., Glynn, P. and Zheng, S. (2016). Analysis of a stochastic approximation algorithm for computing quasi-stationary distributions. Adv. in Appl. Probab.48 792–811. · Zbl 1352.60106
[13] Bouleau, N. and Lépingle, D. (1994). Numerical Methods for Stochastic Processes. Wiley, New York.
[14] Burdzy, K., Hołyst, R. and March, P. (2000). A Fleming–Viot particle representation of the Dirichlet Laplacian. Comm. Math. Phys.214 679–703. · Zbl 0982.60078
[15] Cattiaux, P., Collet, P., Lambert, A., Martínez, S., Méléard, S. and San Martín, J. (2009). Quasi-stationary distributions and diffusion models in population dynamics. Ann. Probab.37 1926–1969. · Zbl 1176.92041
[16] Champagnat, N. and Villemonais, D. (2016). Exponential convergence to quasi-stationary distribution and \(Q\)-process. Probab. Theory Related Fields164 243–283. · Zbl 1334.60015
[17] Cloez, B. and Thai, M.-N. (2016). Fleming–Viot processes: Two explicit examples. ALEA Lat. Am. J. Probab. Math. Stat.13 337–356. · Zbl 1337.60241
[18] Cloez, B. and Thai, M.-N. (2016). Quantitative results for the Fleming–Viot particle system and quasi-stationary distributions in discrete space. Stochastic Process. Appl.126 680–702. · Zbl 1333.60200
[19] Collet, P., Martínez, S., Méléard, S. and San Martín, J. (2011). Quasi-stationary distributions for structured birth and death processes with mutations. Probab. Theory Related Fields151 191–231.
[20] Del Moral, P. and Guionnet, A. (1999). On the stability of measure valued processes with applications to filtering. C. R. Acad. Sci. Paris, Sér. I Math.329 429–434. · Zbl 0935.92001
[21] Del Moral, P. and Miclo, L. (2000). A Moran particle system approximation of Feynman–Kac formulae. Stochastic Process. Appl.86 193–216. · Zbl 1030.65004
[22] Dudley, R. M. (2002). Real Analysis and Probability. Cambridge Studies in Advanced Mathematics74. Cambridge Univ. Press, Cambridge. Revised reprint of the 1989 original. · Zbl 1023.60001
[23] Duflo, M. (2000). Random Iterative Models. Springer, Berlin.
[24] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York. · Zbl 0592.60049
[25] Ferrari, P. A., Kesten, H., Martinez, S. and Picco, P. (1995). Existence of quasi-stationary distributions. A renewal dynamical approach. Ann. Probab.23 501–521. · Zbl 0827.60061
[26] Ferrari, P. A. and Marić, N. (2007). Quasi stationary distributions and Fleming–Viot processes in countable spaces. Electron. J. Probab.12 684–702. · Zbl 1127.60088
[27] Gobet, E. (2000). Weak approximation of killed diffusion using Euler schemes. Stochastic Process. Appl.87 167–197. · Zbl 1045.60082
[28] Gong, G. L., Qian, M. P. and Zhao, Z. X. (1988). Killed diffusions and their conditioning. Probab. Theory Related Fields80 151–167. · Zbl 0631.60073
[29] Grigorescu, I. and Kang, M. (2012). Immortal particle for a catalytic branching process. Probab. Theory Related Fields153 333–361. · Zbl 1251.60064
[30] Lamberton, D. and Pagès, G. (2008). A penalized bandit algorithm. Electron. J. Probab.13 341–373. · Zbl 1206.62139
[31] Lamberton, D., Pagès, G. and Tarrès, P. (2004). When can the two-armed bandit algorithm be trusted? Ann. Appl. Probab.14 1424–1454. · Zbl 1048.62079
[32] Lemaire, V. (2007). An adaptive scheme for the approximation of dissipative systems. Stochastic Process. Appl.117 1491–1518. · Zbl 1126.65005
[33] Lemaire, V. and Menozzi, S. (2010). On some non asymptotic bounds for the Euler scheme. Electron. J. Probab.15 1645–1681. · Zbl 1225.60117
[34] Méléard, S. and Villemonais, D. (2012). Quasi-stationary distributions and population processes. Probab. Surv.9 340–410. · Zbl 1261.92056
[35] Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markovian processes. III. Foster–Lyapunov criteria for continuous-time processes. Adv. in Appl. Probab.25 518–548. · Zbl 0781.60053
[36] Oçafrain, W. and Villemonais, D. (2016). Non-failable approximation method for conditioned distributions. Preprint. Available at arXiv:1606.08978.
[37] Panloup, F. (2008). Recursive computation of the invariant measure of a stochastic differential equation driven by a Lévy process. Ann. Appl. Probab.18 379–426. · Zbl 1136.60049
[38] Pemantle, R. (2007). A survey of random processes with reinforcement. Probab. Surv.4 1–79. · Zbl 1189.60138
[39] Pinsky, R. G. (1995). Positive Harmonic Functions and Diffusion. Cambridge Studies in Advanced Mathematics45. Cambridge Univ. Press, Cambridge. · Zbl 0858.31001
[40] Pollock, M., Fearnhead, P., Johansen, A. M. and Roberts, G. O. (2016). The scalable Langevin exact algorithm: Bayesian inference for big data. Preprint. Available at arXiv:1609.03436.
[41] Rachev, S. T., Klebanov, L. B., Stoyanov, S. V. and Fabozzi, F. J. (2013). The Methods of Distances in the Theory of Probability and Statistics. Springer, New York. · Zbl 1280.60005
[42] Villemonais, D. (2011). Interacting particle systems and Yaglom limit approximation of diffusions with unbounded drift. Electron. J. Probab.16 1663–1692. · Zbl 1244.82052
[43] Villemonais, D. (2014). General approximation method for the distribution of Markov processes conditioned not to be killed. ESAIM Probab. Stat.18 441–467. · Zbl 1310.82032
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.