Bayesian analysis for reversible Markov chains. (English) Zbl 1118.62085

Diaconis, Persi; Rolles, Silke W. W.

Ann. Stat. 34, No. 3, 1270-1292 (2006).

Summary: We introduce a natural conjugate prior for the transition matrix of a reversible Markov chain. This allows estimation and testing. The prior arises from a random walk with reinforcement in the same way the Dirichlet prior arises from the Pólya urn. We give closed form normalizing constants, a simple method of simulation from the posterior and a characterization along the lines of W. E. Johnson’s characterization of the Dirichlet prior [see S. L. Zabell, Ann. Stat. 10, 1091–1099 (1982; Zbl 0512.62007); J. Theor. Probab. 8, No. 1, 175–178 (1995; Zbl 0814.60029)].

MSC:

62M02	Markov processes: hypothesis testing
62F15	Bayesian inference
05C90	Applications of graph theory
60J10	Markov chains (discrete-time Markov processes on discrete state spaces)
62C10	Bayesian problems; characterization of Bayes procedures

Keywords:

reversible Markov chains; conjugate priors; hypothesis testing

Citations:

Zbl 0512.62007; Zbl 0814.60029

References:

[1]	Chan, C. S., Robbins, D. P. and Yuen, D. S. (2000). On the volume of a certain polytope. Experiment. Math. 9 91–99. · Zbl 0960.05004 · doi:10.1080/10586458.2000.10504639
[2]	Dalal, S. R. and Hall, W. J. (1983). Approximating priors by mixtures of natural conjugate priors. J. Roy. Statist. Soc. Ser. B 45 278–286. JSTOR: · Zbl 0532.62017
[3]	Diaconis, P. (1988). Recent progress on de Finetti’s notions of exchangeability. In Bayesian Statistics 3 (J.-M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 111–125. Oxford Univ. Press, New York. · Zbl 0707.60033
[4]	Diaconis, P. and Freedman, D. (1980). de Finetti’s theorem for Markov chains. Ann. Probab. 8 115–130. · Zbl 0426.60064 · doi:10.1214/aop/1176994828
[5]	Diaconis, P. and Ylvisaker, D. (1985). Quantifying prior opinion (with discussion). In Bayesian Statistics 2 (J.-M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 133–156. North-Holland, Amsterdam. · Zbl 0673.62004
[6]	Fortini, S., Ladelli, L., Petris, G. and Regazzini, E. (2002). On mixtures of distributions of Markov chains. Stochastic Process. Appl. 100 147–165. · Zbl 1060.60071 · doi:10.1016/S0304-4149(02)00093-5
[7]	Freedman, D. A. (1962). Mixtures of Markov processes. Ann. Math. Statist. 33 114–118. · Zbl 0112.09902 · doi:10.1214/aoms/1177704716
[8]	Giblin, P. J. (1981). Graphs , Surfaces and Homology. An Introduction to Algebraic Topology , 2nd ed. Chapman and Hall, London. · Zbl 0477.57001
[9]	Good, I. J. (1968). The Estimation of Probabilities. An Essay on Modern Bayesian Methods. MIT Press, Cambridge, MA. · Zbl 0168.39603
[10]	Good, I. J. and Crook, J. F. (1987). The robustness and sensitivity of the mixed-Dirichlet Bayesian test for “independence” in contingency tables. Ann. Statist. 15 670–693. · Zbl 0665.62057 · doi:10.1214/aos/1176350368
[11]	Höglund, T. (1974). Central limit theorems and statistical inference for finite Markov chains. Z. Wahrsch. Verw. Gebiete 29 123–151. · Zbl 0283.60069 · doi:10.1007/BF00532560
[12]	Keane, M. S. (1990). Solution to problem 288. Statist. Neerlandica 44 95–100.
[13]	Keane, M. S. and Rolles, S. W. W. (2000). Edge-reinforced random walk on finite graphs. In Infinite Dimensional Stochastic Analysis (Ph. Clément, F. den Hollander, J. van Neerven and B. de Pagter, eds.) 217–234. R. Neth. Acad. Arts Sci., Amsterdam. · Zbl 0986.05092
[14]	Maurer, S. B. (1976). Matrix generalizations of some theorems on trees, cycles and cocycles in graphs. SIAM J. Appl. Math. 30 143–148. JSTOR: · Zbl 0364.05021 · doi:10.1137/0130017
[15]	Pemantle, R. (1988). Phase transition in reinforced random walk and RWRE on trees. Ann. Probab. 16 1229–1241. · Zbl 0648.60077 · doi:10.1214/aop/1176991687
[16]	Rolles, S. W. W. (2003). How edge-reinforced random walk arises naturally. Probab. Theory Related Fields 126 243–260. · Zbl 1029.60089 · doi:10.1007/s00440-003-0260-8
[17]	Stanley, R. P. (1978). Generating functions. In Studies in Combinatorics (G.-C. Rota, ed.) 100–141. Math. Assoc. Amer., Washington. · Zbl 0422.05003
[18]	Zabell, S. L. (1982). W. E. Johnson’s “sufficientness” postulate. Ann. Statist. 10 1090–1099. · Zbl 0512.62007 · doi:10.1214/aos/1176345975
[19]	Zabell, S. L. (1995). Characterizing Markov exchangeable sequences. J. Theoret. Probab. 8 175–178. · Zbl 0814.60029 · doi:10.1007/BF02213460