×

zbMATH — the first resource for mathematics

Distribution theory for hierarchical processes. (English) Zbl 07036195
Summary: Hierarchies of discrete probability measures are remarkably popular as nonparametric priors in applications, arguably due to two key properties: (i) they naturally represent multiple heterogeneous populations; (ii) they produce ties across populations, resulting in a shrinkage property often described as “sharing of information.” In this paper, we establish a distribution theory for hierarchical random measures that are generated via normalization, thus encompassing both the hierarchical Dirichlet and hierarchical Pitman–Yor processes. These results provide a probabilistic characterization of the induced (partially exchangeable) partition structure, including the distribution and the asymptotics of the number of partition sets, and a complete posterior characterization. They are obtained by representing hierarchical processes in terms of completely random measures, and by applying a novel technique for deriving the associated distributions. Moreover, they also serve as building blocks for new simulation algorithms, and we derive marginal and conditional algorithms for Bayesian inference.

MSC:
60G57 Random measures
62G05 Nonparametric estimation
62F15 Bayesian inference
PDF BibTeX XML Cite
Full Text: DOI Euclid
References:
[1] Blei, D. M., Ng, A. Y. and Jordan, M. I. (2003). Latent Dirichlet allocation. J. Mach. Learn. Res.3 993–1022. · Zbl 1112.68379
[2] Camerlenghi, F., Lijoi, A., Orbanz, P. and Prünster, I. (2019). Distribution theory for hierarchical processes: Supplementary material. DOI:10.1214/17-AOS1678SUPP.
[3] Camerlenghi, F., Lijoi, A. and Prünster, I. (2017). Bayesian prediction with multiple-samples information. J. Multivariate Anal.156 18–28. · Zbl 1369.62116
[4] Caron, F., Teh, Y. W. and Murphy, T. B. (2014). Bayesian nonparametric Plackett–Luce models for the analysis of preferences for college degree programmes. Ann. Appl. Stat.8 1145–1181. · Zbl 06333791
[5] Daley, D. J. and Vere-Jones, D. (2008). An Introduction to the Theory of Point Processes. Vol. II, 2nd ed. Springer, New York. · Zbl 1159.60003
[6] De Blasi, P., Favaro, S., Lijoi, A., Mena, R. H., Ruggiero, M. and Prünster, I. (2015). Are Gibbs-type priors the most natural generalization of the Dirichlet process? IEEE Trans. Pattern Anal. Mach. Intell.37 212–229.
[7] Dunson, D. B. (2010). Nonparametric Bayes applications to biostatistics. In Bayesian Nonparametrics 223–273. Cambridge Univ. Press, Cambridge.
[8] Favaro, S., Lijoi, A., Mena, R. H. and Prünster, I. (2009). Bayesian nonparametric inference for species variety with a two parameter Poisson–Dirichlet process prior. J. Roy. Statist. Soc. Ser. B71 993–1008.
[9] Ferguson, T. S. and Klass, M. J. (1972). A representation of independent increment processes without Gaussian components. Ann. Math. Stat.43 1634–1643. · Zbl 0254.60050
[10] Foti, N. J. and Williamson, S. A. (2015). A survey of non-exchangeable priors for Bayesian nonparametric models. IEEE Trans. Pattern Anal. Mach. Intell.37 359–371.
[11] Gasthaus, J. and Teh, Y. W. (2010). Improvements to the sequence memoizer. Adv. Neuronal Inf. Process. Syst.23.
[12] Gnedin, A. V. and Pitman, J. (2005). Exchangeable Gibbs partitions and Stirling triangles. Zap. Nauchn. Sem. S.-Peterburg. Otdel. Mat. Inst. Steklov. (POMI) 325 83–102. · Zbl 1293.60010
[13] Griffin, J. E., Kolossiatis, M. and Steel, M. F. J. (2013). Comparing distributions by using dependent normalized random-measure mixtures. J. Roy. Statist. Soc. Ser. B75 499–529.
[14] James, L. F. (2005). Bayesian Poisson process partition calculus with an application to Bayesian Lévy moving averages. Ann. Statist.33 1771–1799. · Zbl 1078.62106
[15] James, L. F. (2006). Poisson calculus for spatial neutral to the right processes. Ann. Statist.34 416–440. · Zbl 1091.62012
[16] James, L. F., Lijoi, A. and Prünster, I. (2009). Posterior analysis for normalized random measures with independent increments. Scand. J. Stat.36 76–97. · Zbl 1190.62052
[17] Kallenberg, O. (2005). Probabilistic Symmetries and Invariance Principles. Springer, New York. · Zbl 1084.60003
[18] Kingman, J. F. C. (1967). Completely random measures. Pacific J. Math.21 59–78. · Zbl 0155.23503
[19] Kingman, J. F. C. (1982). The coalescent. Stochastic Process. Appl.13 235–248. · Zbl 0491.60076
[20] Kingman, J. F. C. (1993). Poisson Processes. Oxford Univ. Press, Oxford. · Zbl 0771.60001
[21] Lijoi, A., Mena, R. H. and Prünster, I. (2007). Bayesian nonparametric estimation of the probability of discovering new species. Biometrika94 769–786. · Zbl 1156.62374
[22] Lijoi, A., Mena, R. H. and Prünster, I. (2007). Controlling the reinforcement in Bayesian nonparametric mixture models. J. Roy. Statist. Soc. Ser. B69 715–740.
[23] Lijoi, A., Nipoti, B. and Prünster, I. (2014). Bayesian inference with dependent normalized completely random measures. Bernoulli20 1260–1291. · Zbl 1309.60048
[24] Lijoi, A., Nipoti, B. and Prünster, I. (2014). Dependent mixture models: Clustering and borrowing information. Comput. Statist. Data Anal.71 417–433. · Zbl 06975398
[25] Lijoi, A., Prünster, I. and Walker, S. G. (2008). Bayesian nonparametric estimators derived from conditional Gibbs structures. Ann. Appl. Probab.18 1519–1547. · Zbl 1142.62333
[26] MacEachern, S. N. (1999). Dependent nonparametric processes. In ASA Proceedings of the SBSS 50–55. Amer. Statist. Assoc., Alexandria, VA.
[27] Nguyen, X. (2010). Inference of global clusters from locally distributed data. Bayesian Anal.5 817–846. · Zbl 1330.62255
[28] Nguyen, X. (2016). Borrowing strength in hierarchical Bayes: Convergence of the Dirichlet base measure. Bernoulli22 1535–1571. · Zbl 1360.62103
[29] Pitman, J. (1996). Some developments of the Blackwell–MacQueen urn scheme. In Statistics, Probability and Game Theory (T. S. Ferguson, L. S. Shapley and J. B. MacQueen, eds.). Institute of Mathematical Statistics Lecture Notes—Monograph Series30 245–267. IMS, Hayward, CA.
[30] Pitman, J. (2006). Combinatorial Stochastic Processes. Ecole d’Eté de Probabilités de Saint-Flour XXXII. Lecture Notes in Mathematics N.1875. Springer, Berlin. · Zbl 1103.60004
[31] Pitman, J. and Yor, M. (1997). The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann. Probab.25 855–900. · Zbl 0880.60076
[32] Regazzini, E., Lijoi, A. and Prünster, I. (2003). Distributional results for means of normalized random measures with independent increments. Ann. Statist.31 560–585. · Zbl 1068.62034
[33] Teh, Y. W. and Jordan, M. I. (2010). Hierarchical Bayesian nonparametric models with applications. In Bayesian Nonparametrics 158–207. Cambridge Univ. Press, Cambridge.
[34] Teh, Y. W., Jordan, M. I., Beal, M. J. and Blei, D. M. (2006). Hierarchical Dirichlet processes. J. Amer. Statist. Assoc.101 1566–1581. · Zbl 1171.62349
[35] Walker, S. and Damien, P. (2000). Representations of Lévy processes without Gaussian components. Biometrika87 477–483. · Zbl 1072.60503
[36] Wood, F., Gasthaus, J., Archambeau, C., James, L. F. and Teh, Y. W. (2011). The sequence memoizer. Commun. ACM54 91–98.
[37] Zhou, M. and Carin, L. (2015). Negative binomial process count and mixture modeling. IEEE Trans. Pattern Anal. Mach. Intell.37 307–320.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.