On the number of segregating sites for populations with large family sizes. (English) Zbl 1112.92046

Möhle, M.

Adv. Appl. Probab. 38, No. 3, 750-767 (2006).

Summary: We present recursions for the total number, \(S_n\), of mutations in a sample of \(n\) individuals, when the underlying genealogical tree of the sample is modelled by a coalescent process with mutation rate \(r>0\). The coalescent is allowed to have simultaneous multiple collisions of ancestral lineages, which corresponds to the existence of large families in the underlying population model. For the subclass of \(\Lambda\)-coalescent processes allowing for multiple collisions, such that the measure \(\Lambda(dx)/x\) is finite, we prove that \(S_n/(nr)\) converges in distribution to a limiting variable, \(S\), characterized via an exponential integral of a certain subordinator.
When the measure \(\Lambda (dx)/x^2\) is finite, the distribution of \(S\) coincides with the stationary distribution of an autoregressive process of order 1 and is uniquely determined via a stochastic fixed-point equation of the form \(S\overset \text{D} =AS+B\), with specific independent random coefficients \(A\) and \(B\). Examples are presented in which explicit representations for (the density of) \(S\) are available. We conjecture that \(S_n/E(S_n)\to 1\) in probability if the measure \(\Lambda(dx)/x\) is infinite.

MSC:

92D15	Problems related to evolution
60H30	Applications of stochastic analysis (to PDEs, etc.)
60J27	Continuous-time Markov processes on discrete state spaces
60J20	Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.)
92D10	Genetics and epigenetics
60F05	Central limit and other weak theorems

Keywords:

infinitely-many-sites model; multiple collisions; stochastic difference equation; total number of mutations

References:

[1]	Berestycki, J., Berestycki, N. and Schweinsberg, J. (2005). Small-time behavior of beta-coalescents. · Zbl 1214.60034 · doi:10.1214/07-AIHP103
[2]	Berestycki, J., Berestycki, N. and Schweinsberg, J. (2006). Beta-coalescents and continuous stable random trees. · Zbl 1129.60067 · doi:10.1214/009117906000001114
[3]	Bertoin, J. and Le Gall, J.-F. (2003). Stochastic flows associated to coalescent processes. Prob. Theory Relat. Fields 126 , 261–288. · Zbl 1023.92018 · doi:10.1007/s00440-003-0264-4
[4]	Bertoin, J. and Yor, M. (2001). On subordinators, self-similar Markov processes and some factorizations of the exponential variable. Electron. Commun. Prob. 6 , 95–106. · Zbl 1024.60030
[5]	Bolthausen, E. and Sznitman, A.-S. (1998). On Ruelle’s probability cascades and an abstract cavity method. Commun. Math. Phys. 197 , 247–276. · Zbl 0927.60071 · doi:10.1007/s002200050450
[6]	Brandt, A. (1986). The stochastic equation \(Y_n+1=A_n Y_n+B_n\) with stationary coefficients. Adv. Appl. Prob. 18 , 211–220. JSTOR: · Zbl 0588.60056 · doi:10.2307/1427243
[7]	Carmona, P., Petit, F. and Yor, M. (1997). On the distribution and asymptotic results for exponential functionals of Lévy processes. In Exponential Functionals and Principal Values Related to Brownian Motion , ed. M. Yor, Biblioteca de la Revista Matematica Iberoamericana, Madrid, pp. 73–121. · Zbl 0905.60056
[8]	Durrett, R. and Schweinsberg, J. (2005). A coalescent model for the effect of advantageous mutations on the genealogy of a population. Stoch. Process. Appl. 115 , 1628–1657. · Zbl 1082.92031 · doi:10.1016/j.spa.2005.04.009
[9]	Goldschmidt, C. and Martin, J. B. (2005). Random recursive trees and the Bolthausen–Sznitman coalescent. Electron. J. Prob. 10 , 718–745. · Zbl 1109.60060
[10]	Hedgecock, D. (1994). Does variance in reproductive success limit effective population sizes of marine organisms? In Genetics and Evolution of Aquatic Organisms , ed. A. Beaumont, Chapman and Hall, London, pp. 122–134.
[11]	Kingman, J. F. C. (1982). The coalescent. Stoch. Process. Appl. 13 , 235–248. · Zbl 0491.60076 · doi:10.1016/0304-4149(82)90011-4
[12]	Möhle, M. and Sagitov, S. (2001). A classification of coalescent processes for haploid exchangeable population models. Ann. Prob. 29 , 1547–1562. · Zbl 1013.92029 · doi:10.1214/aop/1015345761
[13]	Neininger, R. and Rüschendorf, L. (2004). On the contraction method with degenerate limit equation. Ann. Prob. 32 , 2838–2856. · Zbl 1060.60005 · doi:10.1214/009117904000000171
[14]	Pitman, J. (1999). Coalescents with multiple collisions. Ann. Prob. 27 , 1870–1902. · Zbl 0963.60079 · doi:10.1214/aop/1022677552
[15]	Rösler, U. (1991). A limit theorem for ‘Quicksort’. RAIRO Inf. Théoret. Appl. 25 , 85–100. · Zbl 0718.68026
[16]	Rösler, U. (1992). A fixed point theorem for distributions. Stoch. Process. Appl. 42 , 195–214. · Zbl 0761.60015 · doi:10.1016/0304-4149(92)90035-O
[17]	Rösler, U. and Rüschendorf, L. (2001). The contraction method for recursive algorithms. Algorithmica 29 , 3–33. · Zbl 0967.68166 · doi:10.1007/s004530010053
[18]	Sagitov, S. (1999). The general coalescent with asynchronous mergers of ancestral lines. J. Appl. Prob. 36 , 1116–1125. · Zbl 0962.92026 · doi:10.1239/jap/1032374759
[19]	Schweinsberg, J. (2000). Coalescents with simultaneous multiple collisions. Electron. J. Prob. 5 , 1–50. · Zbl 0959.60065
[20]	Vervaat, W. (1979). On a stochastic difference equation and a representation of non-negative infinitely divisible random variables. Adv. Appl. Prob. 11 , 750–783. JSTOR: · Zbl 0417.60073 · doi:10.2307/1426858