×

Predictive inference with Fleming-Viot-driven dependent Dirichlet processes. (English) Zbl 1475.62229

Summary: We consider predictive inference using a class of temporally dependent Dirichlet processes driven by Fleming-Viot diffusions, which have a natural bearing in Bayesian nonparametrics and lend the resulting family of random probability measures to analytical posterior analysis. Formulating the implied statistical model as a hidden Markov model, we fully describe the predictive distribution induced by these Fleming-Viot-driven dependent Dirichlet processes, for a sequence of observations collected at a certain time given another set of draws collected at several previous times. This is identified as a mixture of Pólya urns, whereby the observations can be values from the baseline distribution or copies of previous draws collected at the same time as in the usual Pólya urn, or can be sampled from a random subset of the data collected at previous times. We characterize the time-dependent weights of the mixture which select such subsets and discuss the asymptotic regimes. We describe the induced partition by means of a Chinese restaurant process metaphor with a conveyor belt, whereby new customers who do not sit at an occupied table open a new table by picking a dish either from the baseline distribution or from a time-varying offer available on the conveyor belt. We lay out explicit algorithms for exact and approximate posterior sampling of both observations and partitions, and illustrate our results on predictive problems with synthetic and real data.

MSC:

62M05 Markov processes: estimation; hidden Markov models
60G57 Random measures
62F15 Bayesian inference
62G05 Nonparametric estimation
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Antoniak, C. E. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Statist. 2, 1152-1174. · Zbl 0335.60034 · doi:10.1214/aos/1176342871
[2] Ascolani, F., Lijoi, A. and Ruggiero, M. (2020). Predictive inference with Fleming-Viot-driven dependent Dirichlet processes - Supplementary Material. Bayesian Analysis. · Zbl 1475.62229 · doi:10.1214/20-BA1206SUPP
[3] Barrientos, A. F., Jara, A. and Quintana, F.A. (2012). On the support of MacEachern’s dependent Dirichlet processes and extensions. Bayes. Anal. 7, 277-310. · Zbl 1330.60067 · doi:10.1214/12-BA709
[4] Beal, M. J., Ghahramani, Z. and Rasmussen, C.E. (2002). The infinite hidden Markov model. Advances in Neural Information Processing Systems 14, 577-585.
[5] Canale, A. and Ruggiero, M. (2016). Bayesian nonparametric forecasting of monotonic functional time series. Electron. J. Stat. 10, 3265-3286. · Zbl 1357.62278 · doi:10.1214/16-EJS1190
[6] Caron, F., Davy, M. and Doucet, A. (2007) Generalized Pólya urn for time-varying Dirichlet process mixtures. Proc. 23rd Conf. on Uncertainty in Artificial Intelligence, Vancouver.
[7] Caron, F., Neiswanger, W., Wood, F., Doucet, A. and Davy, M. (2017). Generalized Pólya urn for time-varying Pitman-Yor processes. J. Mach. Learn. Res. 18, 1-32. · Zbl 1437.62229
[8] Caron, F. and Teh, Y. W.(2012). Bayesian nonparametric models for ranked data. Neural Information Processing Systems (NIPS 2012), Lake Tahoe, USA, 2012.
[9] De Blasi, P., Favaro, S., Lijoi, A., Mena, R. H., Prünster, I. and Ruggiero, M. (2015). Are Gibbs-type priors the most natural generalization of the Dirichlet process? IEEE Trans. Pattern Anal. Mach. Intell. 37, 212-229. · doi:10.1109/TPAMI.2013.217
[10] Dunson, D. B. (2006). Bayesian dynamic modeling of latent trait distributions. Biostatistics 7, 551-568. · Zbl 1170.62375 · doi:10.1093/biostatistics/kxj025
[11] Ethier, S. N. and Griffiths, R. C. (1993). The transition function of a Fleming-Viot process. Ann. Probab. 21, 1571-1590. · Zbl 0778.60038 · doi:10.1214/aop/1176989131
[12] Ethier, S. N. and Kurtz, T. G. (1993). Fleming-Viot processes in population genetics. SIAM J. Control Optim. 31, 345-386. · Zbl 0774.60045 · doi:10.1137/0331019
[13] Favaro, S., Ruggiero, M. and Walker, S. G. (2009). On a Gibbs sampler based random process in Bayesian nonparametrics. Electron. J. Stat. 3, 1556-1566. · Zbl 1326.60105 · doi:10.1214/09-EJS563
[14] Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1, 209-230. · Zbl 0255.62037 · doi:10.1214/aos/1176342360
[15] Ghosal, S. and van der Vaart, A. (2017). Fundamentals of Nonparametric Bayesian Inference. Cambridge University Press. · Zbl 1376.62004 · doi:10.1017/9781139029834
[16] Gnedin, A. and Pitman, J. (2005). Exchangeable Gibbs partitions and Stirling triangles. Zap. Nauchn. Sem. S.-Peterburg. Otdel. Mat. Inst. Steklov. (POMI) 325, 83-102. · Zbl 1293.60010 · doi:10.1007/s10958-006-0335-z
[17] Griffin, J. E. and Steel, M. F. J. (2010). Stick-breaking autoregressive processes. J. Econometrics 162, 383-396. · Zbl 1441.62709 · doi:10.1016/j.jeconom.2011.03.001
[18] Griffiths, R. C. (1980). Lines of descent in the diffusion approximation of neutral Wright-Fisher models. Theor. Popul. Biol. 17, 37-50. · Zbl 0434.92011 · doi:10.1016/0040-5809(80)90013-1
[19] Gutierrez, L., Mena, R. H. and Ruggiero, M. (2016). A time dependent Bayesian nonparametric model for air quality analysis. Comput. Statist. Data Anal. 95, 161-175. · Zbl 1468.62071 · doi:10.1016/j.csda.2015.10.002
[20] Hjort, N. L., Holmes, C. C., Müller, P. and Walker, S.G., eds. (2010). Bayesian Nonparametrics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge Univ. Press. · doi:10.1017/CBO9780511802478.002
[21] Jenkins, P. A. and Spanò, D. (2017). Exact simulation of the Wright-Fisher diffusion. Ann. Appl. Probab. 3, 1478-1509. · Zbl 1385.65006 · doi:10.1214/16-AAP1236
[22] Kon Kam King, G., Canale, A. and Ruggiero, M. (2020). Bayesian functional forecasting with locally-autoregressive dependent processes. Bayesian Anal. 14, 1121-1141. · Zbl 1435.62348 · doi:10.1214/18-BA1140
[23] Kon Kam King, G., Papaspiliopoulos, O. and Ruggiero, M. (2020). Exact inference for a class of hidden Markov models on general state spaces. Preprint. · Zbl 1471.62449
[24] Klein, J. P. and Moeschberger, M. L. (1997). Survival Analysis Techniques for Censored and Truncated Data. Springer-Verlag New York · Zbl 0871.62091
[25] Lavine, M. (1992). Some aspects of Polya tree distributions for statistical modelling. Ann. Statist. 20, 1222-1235. · Zbl 0765.62005 · doi:10.1214/aos/1176348767
[26] Lijoi, A., Mena, R. H. and Prünster, I. (2005). Hierarchical mixture modeling with normalized inverse-gaussian priors. J. Amer. Statist. Assoc. 472, 1278-1291. · Zbl 1117.62386 · doi:10.1198/016214505000000132
[27] Lijoi, A., Mena, R. H. and Prünster, I. (2007). Controlling the reinforcement in Bayesian non-parametric mixture models. J. R. Statist. Soc. Ser. B 69, 715-740. · Zbl 07555373 · doi:10.1111/j.1467-9868.2007.00609.x
[28] Lijoi, A., Ruggiero, M. and Spanò, D. (2016). On the transition function of some time-dependent Dirichlet and gamma processes. In JSM Proceedings, Section on Nonparametric Statistics. Alexandria, VA: American Statistical Association.
[29] Lo, A. Y. (1984). On a class of Bayesian nonparametric estimates: I. Density estimates. Ann. Statist. 12, 351-357. · Zbl 0557.62036 · doi:10.1214/aos/1176346412
[30] MacEachern, S. N. (1999). Dependent nonparametric Processes. In ASA Proceedings of the Section on Bayesian Statistical Science. American Statist. Assoc., Alexandria, VA.
[31] Mauldin, R. D., Sudderth, W. D. and Williams, S. C. (1992). Polya trees and random distributions. Ann. Statist. 20, 1203-1221. · Zbl 0765.62006 · doi:10.1214/aos/1176348766
[32] Mena, R. H. and Ruggiero, M. (2016). Dynamic density estimation with diffusive Dirichlet mixtures. Bernoulli 22, 901-926. · Zbl 1388.62099 · doi:10.3150/14-BEJ681
[33] Müller, P., Quintana, F. A., Jara, A. and Hanson, T. (2015). Bayesian Nonparametric Data Analysis. Springer. · Zbl 1333.62003 · doi:10.1007/978-3-319-18968-0
[34] Papaspiliopoulos, O. and Ruggiero, M. (2014). Optimal filtering and the dual process. Bernoulli 20, 1999-2019. · Zbl 1302.60071 · doi:10.3150/13-BEJ548
[35] Papaspiliopoulos, O., Ruggiero, M. and Spanò, D. (2016). Conjugacy properties of time-evolving Dirichlet and gamma random measures. Electron. J. Stat. 10, 3452-3489. · Zbl 1353.62092 · doi:10.1214/16-EJS1194
[36] Perman, M., Pitman, J. and Yor, M. (1992). Size-biased sampling of Poisson point processes and excursions. Probab. Theory Related Fields 92, 21-39. · Zbl 0741.60037 · doi:10.1007/BF01205234
[37] Pitman, J. (2006). Combinatorial Stochastic Processes. Lecture Notes in Math. 1875. Springer, Berlin. · Zbl 1103.60004
[38] Pitman, J. and Yor, M. (1997). The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Ann. Probab. 25, 855-900. · Zbl 0880.60076 · doi:10.1214/aop/1024404422
[39] Rodriguez, A. and ter Horst, E. (2008). Bayesian dynamic density estimation. Bayes. Anal. 3, 339-366. · Zbl 1330.62180 · doi:10.1214/08-BA313
[40] Regazzini, E., Lijoi, A. and Prünster, I. (2003). Distributional results for means of random measures with independent increments. Ann. Statist. 31, 560-585. · Zbl 1068.62034 · doi:10.1214/aos/1051027881
[41] Sethuraman, J. (1994). A constructive definition of the Dirichlet process prior. Statist. Sinica 2, 639-650. · Zbl 0823.62007
[42] Stepleton, T., Ghahramani, Z., Gordon, G., and Lee, T.-S. (2009). The block diagonal infinite hidden Markov model. Journal of Machine Learning Research 5, 544-551.
[43] Tavaré, S. (1984). Line-of-descent and genealogical processes, and their applications in population genetic models. Theoret. Population Biol. 26, 119-164. · Zbl 0555.92011 · doi:10.1016/0040-5809(84)90027-3
[44] Van Gael, V., Saatci, Y., Teh, Y. W. and Ghahramani, Z. (2008). Beam sampling for the infinite hidden Markov model. In Proceedings of the 25th International Conference on Machine Learning.
[45] Walker, S. G., Hatjispyros S. J. and Nicoleris, T. (2007). A Fleming-Viot process and Bayesian nonparametrics. Ann. Appl. Probab. 17, 67-80. · Zbl 1131.60045 · doi:10.1214/105051606000000600
[46] Yau, C., Papaspiliopoulos, O., Roberts, G. O. and Holmes, C. (2011). Bayesian non-parametric hidden Markov models with applications in genomics. J. Roy. Statist. Soc. Ser. B 73, 37-57. · Zbl 1411.62247 · doi:10.1111/j.1467-9868.2010.00756.x
[47] Zhang, A., Zhu, J. and Zhang, B. (2014). Max-margin infinite hidden Markov models. In Proceedings of the 31st International Conference on Machine Learning
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.