×

Hierarchical species sampling models. (English) Zbl 1459.62039

Summary: This paper introduces a general class of hierarchical nonparametric prior distributions which includes new hierarchical mixture priors such as the hierarchical Gnedin measures, and other well-known prior distributions such as the hierarchical Pitman-Yor and the hierarchical normalized random measures. The random probability measures are constructed by a hierarchy of generalized species sampling processes with possibly non-diffuse base measures. The proposed framework provides a probabilistic foundation for hierarchical random measures, and allows for studying their properties under the alternative assumptions of diffuse, atomic and mixed base measure. We show that hierarchical species sampling models have a Chinese Restaurants Franchise representation and can be used as prior distributions to undertake Bayesian nonparametric inference. We provide a general sampling method for posterior approximation which easily accounts for non-diffuse base measures such as spike-and-slab.

MSC:

62G05 Nonparametric estimation
60G57 Random measures
60G09 Exchangeability for stochastic processes
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Argiento, R., Cremaschi, A., and Vannucci, M. (2019). “Hierarchical Normalized Completely Random Measures to Cluster Grouped Data.” Journal of the American Statistical Association, 1-43. · Zbl 1435.62121 · doi:10.1080/01621459.2019.1594833
[2] Arratia, R., Barbour, A. D., and S., T. (2003). Logarithmic combinatorial structures: a probabilistic approach. European Mathematical Society. · Zbl 1040.60001
[3] Bacallado, S., Battiston, M., Favaro, S., and Trippa, L. (2017). “Sufficientness Postulates for Gibbs-Type Priors and Hierarchical Generalizations.” Statistical Science, 32(4): 487-500. · Zbl 1383.62079 · doi:10.1214/17-STS619
[4] Bassetti, F., Casarin, R., and Leisen, F. (2014). “Beta-product dependent Pitman-Yor processes for Bayesian inference.” Journal of Econometrics, 180(1): 49-72. · Zbl 1298.62148 · doi:10.1016/j.jeconom.2014.01.007
[5] Bassetti, F., Casarin, R., Rossini, L. (2019a). “Supplementary Material A to Hierarchical Species Sampling Models.” Bayesian Analysis.
[6] Bassetti, F., Casarin, R., Rossini, L. (2019b). “Supplementary Material B to Hierarchical Species Sampling Models.” Bayesian Analysis.
[7] Billio, M., Casarin, R., and Rossini, L. (2019). “Bayesian nonparametric sparse VAR models.” Journal of Econometrics, 212: 97-115. URL http://www.sciencedirect.com/science/article/pii/S0304407619300776. · Zbl 1452.62883 · doi:10.1016/j.jeconom.2019.04.022
[8] Camerlenghi, F., Lijoi, A., Orbanz, P., and Pruenster, I. (2019). “Distribution theory for hierarchical processes.” Annals of Statistics, 47(1): 67-92. · Zbl 1478.60151 · doi:10.1214/17-AOS1678
[9] Camerlenghi, F., Lijoi, A., and Prünster, I. (2017). “Bayesian prediction with multiple-samples information.” Journal of Multivariate Analysis, 156: 18-28. URL http://www.sciencedirect.com/science/article/pii/S0047259X17300568. · Zbl 1369.62116 · doi:10.1016/j.jmva.2017.01.010
[10] Camerlenghi, F., Lijoi, A., and Prünster, I. (2018). “Bayesian nonparametric inference beyond the Gibbs-type framework.” Scandinavian Journal of Statistics, 45(4): 1062-1091. · Zbl 1408.62052 · doi:10.1111/sjos.12334
[11] Canale, A., Lijoi, A., Nipoti, B., and Prünster, I. (2017). “On the Pitman-Yor process with spike and slab base measure.” Biometrika, 104(3): 681-697. · Zbl 07072235 · doi:10.1093/biomet/asx041
[12] Castillo, I., Schmidt-Hieber, J., and van der Vaart, A. (2015). “Bayesian linear regression with sparse priors.” Annals of Statistics, 43(5): 1986-2018. URL https://projecteuclid.org:443/euclid.aos/1438606851. · Zbl 1486.62197 · doi:10.1214/15-AOS1334
[13] Dahl, D. B. (2006). “Model-based clustering for expression data via a Dirichlet process mixture model.” In Do, K.-A., Müller, P. P., and Vannucci, M. (eds.), Bayesian Inference for Gene Expression and Proteomics, 201-218. Cambridge University Press.
[14] De Blasi, P., Favaro, S., Lijoi, A., Mena, R. H., Prunster, I., and Ruggiero, M. (2015). “Are Gibbs-Type Priors the Most Natural Generalization of the Dirichlet Process?” IEEE Transactions on Pattern Analysis & Machine Intelligence, 37(2): 212-229.
[15] Diaconis, P. and Ram, A. (2012). “A probabilistic interpretation of the Macdonald polynomials.” Annals of Probability, 40(5): 1861-1896. · Zbl 1255.05194 · doi:10.1214/11-AOP674
[16] Donnelly, P. (1986). “Partition structures, Pólya urns, the Ewens sampling formula, and the ages of alleles.” Theoretical Population Biology, 30(2): 271-288. · Zbl 0608.92005 · doi:10.1016/0040-5809(86)90037-7
[17] Donnelly, P. and Grimmett, G. (1993). “On the asymptotic distribution of large prime factors.” Journal of the London Mathematical Society (2), 47(3): 395-404. · Zbl 0839.11039 · doi:10.1112/jlms/s2-47.3.395
[18] Du, L., Buntine, W., and Jin, H. (2010). “A segmented topic model based on the two-parameter Poisson-Dirichlet process.” Machine Learning, 81(1): 5-19. · Zbl 1470.68100
[19] Dubey, A., Williamson, S., and Xing, E. (2014). “Parallel Markov chain Monte Carlo for Pitman-Yor mixture models.” In Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014, 142-151.
[20] Escobar, M. (1994). “Estimating normal means with a Dirichlet process prior.” Journal of the American Statistical Association, 89(425): 268-277. · Zbl 0791.62039 · doi:10.1080/01621459.1994.10476468
[21] Escobar, M. and West, M. (1995). “Bayesian density estimation and inference using mixtures.” Journal of the American Statistical Association, 90(430): 577-588. · Zbl 0826.62021 · doi:10.1080/01621459.1995.10476550
[22] Ewens, W. J. (1972). “The sampling theory of selectively neutral alleles.” Theoretical Population Biology, 3: 87-112; erratum, ibid. 3 (1972), 240; erratum, ibid. 3 (1972), 376. · Zbl 0245.92009 · doi:10.1016/0040-5809(72)90035-4
[23] Favaro, S. and Teh, Y. W. (2013). “MCMC for Normalized Random Measure Mixture Models.” Statistical Science, 28(3): 335-359. · Zbl 1331.62138 · doi:10.1214/13-STS422
[24] George, E. I. and McCulloch, R. E. (1993). “Variable Selection via Gibbs Sampling.” Journal of the American Statistical Association, 88(423): 881-889. URL http://www.tandfonline.com/doi/abs/10.1080/01621459.1993.10476353
[25] Gnedin, A. (2010). “A species sampling model with finitely many types.” Electronic Communications in Probability, 15(8): 79-88. · Zbl 1202.60056 · doi:10.1214/ECP.v15-1532
[26] Gnedin, A. and Pitman, J. (2006). “Exchangeable Gibbs partitions and Stirling triangles.” Journal of Mathematical Sciences, 138(3): 5674-5685. · Zbl 1293.60010
[27] Griffin, J. E. and Steel, M. F. J. (2011). “Stick-breaking autoregressive processes.” Journal of Econometrics, 162(2): 383-396. · Zbl 1441.62709 · doi:10.1016/j.jeconom.2011.03.001
[28] Hirano, K. (2002). “Semiparametric Bayesian Inference in autoregressive panel data models.” Econometrica, 70(2): 781-799. · Zbl 1121.62557 · doi:10.1111/1468-0262.00305
[29] Hjort, N. L., Homes, C., Müller, P., and Walker, S. G. (2010). Bayesian Nonparametrics. Cambridge University Press. · Zbl 1192.62080
[30] Hoppe, F. M. (1984). “Pólya-like urns and the Ewens’ sampling formula.” Journal of Mathematical Biology, 20(1): 91-94. · Zbl 0547.92009 · doi:10.1007/BF00275863
[31] Kallenberg, O. (2006). Probabilistic Symmetries and Invariance Principles. Springer-Verlag New York. · Zbl 1084.60003
[32] Kalli, M. and Griffin, J. E. (2018). “Bayesian nonparametric vector autoregressive models.” Journal of Econometrics, 203(2): 267-282. URL http://www.sciencedirect.com/science/article/pii/S0304407617302415. · Zbl 1386.62075 · doi:10.1016/j.jeconom.2017.11.009
[33] Kalli, M., Griffin, J. E., and Walker, S. (2011). “Slice sampling mixture models.” Statistics and Computing, 21(1): 93-105. · Zbl 1256.65006 · doi:10.1007/s11222-009-9150-y
[34] Kim, S., Dahl, D. B., and Vannucci, M. (2009). “Spiked Dirichlet process prior for Bayesian multiple hypothesis testing in random effects models.” Bayesian Analysis, 4(4): 707-732. · Zbl 1330.62029 · doi:10.1214/09-BA426
[35] Kingman, J. F. C. (1980). Mathematics of genetic diversity, volume 34 of CBMS-NSF Regional Conference Series in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, Pa. · Zbl 0458.92009
[36] Lau, J. W. and Green, P. J. (2007). “Bayesian Model-Based Clustering Procedures.” Journal of Computational and Graphical Statistics, 16(3): 526-558.
[37] Lim, K. W., Buntine, W., Chen, C., and Du, L. (2016). “Nonparametric Bayesian topic modelling with the hierarchical Pitman-Yor processes.” International Journal of Approximate Reasoning, 78(C): 172-191. · Zbl 06639853 · doi:10.1016/j.ijar.2016.07.007
[38] Miller, J. and Harrison, M. (2018). “Mixture models with a Prior on the number of components.” Journal of the American Statistical Association, 113(521): 340-356. · Zbl 1398.62066 · doi:10.1080/01621459.2016.1255636
[39] Müller, P. and Quintana, F. (2010). “Random partition models with regression on covariates.” Journal of Statistical Planning and Inference, 140(10): 2801-2808. · Zbl 1191.62073 · doi:10.1016/j.jspi.2010.03.002
[40] Navarro, D. J., Griffiths, T. L., Steyvers, M., and Lee, M. D. (2006). “Modeling individual differences using Dirichlet processes.” Journal of Mathematical Psychology, 50(2): 101-122. · Zbl 1138.91594 · doi:10.1016/j.jmp.2005.11.006
[41] Neal, R. (2000). “Markov Chain sampling methods for Dirichlet process mixture models.” Journal of Computational and Graphical Statistics, 9(2): 249-265.
[42] Nguyen, X. (2016). “Borrowing strengh in hierarchical Bayes: Posterior concentration of the Dirichlet base measure.” Bernoulli, 22(3): 1535-1571. · Zbl 1360.62103 · doi:10.3150/15-BEJ703
[43] Papaspiliopoulos, O. and Roberts, G. O. (2008). “Retrospective Markov Chain Monte Carlo Methods for Dirichlet Process Hierarchical Models.” Biometrika, 95(1): 169-186. · Zbl 1437.62576 · doi:10.1093/biomet/asm086
[44] Pitman, J. (1995). “Exchangeable and partially exchangeable random partitions.” Probability Theory and Related Fields, 102(2): 145-158. · Zbl 0821.60047 · doi:10.1007/BF01213386
[45] Pitman, J. (1996). “Some developments of the Blackwell-MacQueen urn scheme.” In Statistics, probability and game theory, volume 30 of IMS Lecture Notes—Monograph Series, 245-267. Institute of Mathematical Statistics, Hayward, CA. · Zbl 0996.60500
[46] Pitman, J. (2003). “Poisson-Kingman partitions.” In Statistics and science: a Festschrift for Terry Speed, volume 40 of IMS Lecture Notes—Monograph Series, 1-34. Institute of Mathematical Statistics, Beachwood, OH. · Zbl 1029.00084
[47] Pitman, J. (2006). Combinatorial Stochastic Processes, volume 1875. Springer-Verlag. · Zbl 1103.60004
[48] Pitman, J. and Yor, M. (1997). “The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator.” The Annals of Probability, 25(2): 855-900. · Zbl 0880.60076 · doi:10.1214/aop/1024404422
[49] Rockova, V. and George, E. I. (2018). “The Spike-and-Slab LASSO.” Journal of the American Statistical Association, 113(521): 431-444. · Zbl 1398.62186 · doi:10.1080/01621459.2016.1260469
[50] Sangalli, L. M. (2006). “Some developments of the normalized random measures with independent increments.” Sankhyā, 68(3): 461-487. · Zbl 1193.62038
[51] Sohn, K.-A. and Xing, E. P. (2009). “A hierarchical Dirichlet process mixture model for haplotype reconstruction from multi-population data.” The Annals of Applied Statistics, 3(2): 791-821. · Zbl 1166.62085 · doi:10.1214/08-AOAS225
[52] Stock, J. H. and Watson, M. W. (2002). “Forecasting Using Principal Components from a Large Number of Predictors.” Journal of the American Statistical Association, 97(460): 1167-1179. · Zbl 1041.62081 · doi:10.1198/016214502388618960
[53] Teh, Y. and Jordan, M. I. (2010). “Hierarchical Bayesian nonparametric models with applications.” In Hjort, N. L., Holmes, C., Müller, P., and Walker, S. (eds.), Bayesian Nonparametrics. Cambridge University Press.
[54] Teh, Y. W. (2006). “A Hierarchical Bayesian Language Model Based on Pitman-Yor Processes.” In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, ACL-44, 985-992. Stroudsburg, PA, USA: Association for Computational Linguistics.
[55] Teh, Y. W., Jordan, M. I., Beal, M. J., and Blei, D. M. (2006). “Hierarchical Dirichlet processes.” Journal of the American Statistical Association, 101(476): 1566-1581. · Zbl 1171.62349 · doi:10.1198/016214506000000302
[56] Walker, S. G. (2007). “Sampling the Dirichlet Mixture Model with Slices.” Communications in Statistics - Simulation and Computation, 36(1): 45-54. · Zbl 1113.62058 · doi:10.1080/03610910601096262
[57] Wood, F.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.