×

zbMATH — the first resource for mathematics

Sampling hierarchies of discrete random structures. (English) Zbl 1452.62253
Summary: Hierarchical normalized discrete random measures identify a general class of priors that is suited to flexibly learn how the distribution of a response variable changes across groups of observations. A special case widely used in practice is the hierarchical Dirichlet process. Although current theory on hierarchies of nonparametric priors yields all relevant tools for drawing posterior inference, their implementation comes at a high computational cost. We fill this gap by proposing an approximation for a general class of hierarchical processes, which leads to an efficient conditional Gibbs sampling algorithm. The key idea consists of a deterministic truncation of the underlying random probability measures leading to a finite dimensional approximation of the original prior law. We provide both empirical and theoretical support for such a procedure.
MSC:
62G05 Nonparametric estimation
60G55 Point processes (e.g., Poisson, Cox, Hawkes processes)
Software:
BNPdensity; Stan
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Arbel, J.; Prünster, I., A moment-matching Ferguson & Klass algorithm, Stat. Comput., 27, 1, 3-17 (2017) · Zbl 06696848
[2] Arbel, J.; De Blasi, P.; Prünster, I., Stochastic approximations to the Pitman-Yor process, Bayesian Anal., 14, 4, 1201-1219 (2019) · Zbl 1435.62076
[3] Argiento, R.; Bianchini, I.; Guglielmi, A., A blocked gibbs sampler for ngg-mixture models via a priori truncation, Stat. Comput., 26, 3, 641-666 (2016) · Zbl 06591373
[4] Argiento, R.; Cremaschi, A.; Vannucci, M., Hierarchical normalized completely random measures to cluster grouped data, J. Am. Stat. Assoc., 115, 229, 318-333 (2020) · Zbl 1437.62224
[5] Barrios, E.; Lijoi, A.; Nieto-Barajas, LE; Prünster, I., Modeling with normalized random measure mixture models, Stat. Sci., 28, 3, 313-334 (2013) · Zbl 1331.62120
[6] Bassetti, F.; Casarin, R.; Rossini, L., Hierarchical species sampling models, Bayesian Anal. (2020)
[7] Blei, DM; Ng, AY; Jordan, MI, Latent Dirichlet allocation, J. Mach. Learn. Res., 3, 993-1022 (2003) · Zbl 1112.68379
[8] Camerlenghi, F.; Lijoi, A.; Orbanz, P.; Prünster, I., Distribution theory for hierarchical processes, Ann. Stat., 49, 1, 67-92 (2019) · Zbl 07036195
[9] Carlton, MA, A family of densities derived from the three-parameter Dirichlet process, J. Appl. Prob., 39, 764-774 (2002) · Zbl 1014.60053
[10] Carpenter, B.; Gelman, A.; Hoffman, M.; Lee, D.; Goodrich, B.; Betancourt, M.; Brubaker, M.; Guo, J.; Li, P.; Riddel, A., Stan: a probabilistic programming language, J. Stat. Softw., 76, 1, 1-32 (2017)
[11] Cifarelli, D.; Regazzini, E., Problemi statistici non parametrici in condizioni di scambiabilità parziale, Technical report, Quaderni Istituto Matematica Finanziaria, Università di Torino Serie, III, 12 (1978)
[12] Clogg, CC; Goodman, LA, On scaling models applied to data from several groups, Psychometrika, 51, 1, 123-135 (1986)
[13] Connor, RJ; Mosimman, JE, Concepts of independence for proportions with a generalization of the Dirichlet distribution, J. Am. Stat. Assoc., 64, 325, 194-206 (1969)
[14] Daley, DJ., Vere-Jones, D.: An introduction to the theory of point processes. Vol. I, 2nd edn. Probability and its Applications (New York), Springer, New York, elementary theory and methods (2003) · Zbl 1026.60061
[15] De Blasi, P.; Favaro, S.; Lijoi, A.; Mena, RH; Prünster, I.; Ruggiero, M., Are Gibbs-type priors the most natural generalization of the dirichlet process?, IEEE Trans. Pattern Anal. Mach. Intell., 37, 2, 212-229 (2015)
[16] Diaconis, P.; Ylvisaker, D., Conjugate prior for exponential families, Ann. Stat., 7, 2, 269-292 (1979) · Zbl 0405.62011
[17] Dunson, DB; Xing, C., Nonparametric Bayes modeling of multivariate categorical data, J. Am. Stat. Assoc., 104, 487, 1042-1051 (2009) · Zbl 1388.62151
[18] Escobar, MD; West, M., Bayesian density estimation and inference using mixtures, J. Am. Stat. Assoc., 90, 430, 577-588 (1995) · Zbl 0826.62021
[19] Favaro, S.; Hadjicharalambous, G.; Prünster, I., On a class of distributions on the simplex, J. Stat. Plan. Inference, 141, 9, 2987-3004 (2011) · Zbl 1215.62011
[20] Ferguson, TS, A Bayesian analysis of some nonparametric problems, Ann. Stat., 1, 2, 209-230 (1973) · Zbl 0255.62037
[21] Ferguson, TS; Klass, MJ, A representation of independent increment processes without Gaussian components, Ann. Math. Stat., 43, 5, 1634-1643 (1972) · Zbl 0254.60050
[22] Fox, EB; Sudderth, EB; Jordan, MI; Willsky, AS, A sticky HDP-HMM with application to speaker diarization, Ann. Appl. Stat., 5, 2, 1020-1056 (2011) · Zbl 1232.62077
[23] Goodman, LA, Exploratory latent structure analysis using both identifiable and unindentifiable models, Biometrika, 61, 2, 215-231 (1974) · Zbl 0281.62057
[24] Goodman, LA, A new model for scaling response patterns: an application of quasi independence concept, J. Am. Stat. Assoc., 70, 352, 755-768 (1975) · Zbl 0321.62056
[25] Griffin, J.E., Leisen, F.: Compound random measures and their use in Bayesian non-parametrics. J. R. Stat. Soc. Series B Stat. Methodol. 79(2), 525-545 (2017) · Zbl 1412.60071
[26] Hagenaars, JA; McCutcheon, AL, Applied Latent Class Analysis (2002), Cambridge: Cambridge University Press, Cambridge · Zbl 1003.00021
[27] Ishwaran, H.; James, LF, Gibbs sampling methods for stick-breaking priors, J. Am. Stat. Assoc., 96, 453, 161-173 (2001) · Zbl 1014.62006
[28] Ishwaran, H.; Zarepour, M., Exact and approximate sum representations for the Dirichlet process, Can. J. Stat., 30, 2, 269-283 (2002) · Zbl 1035.60048
[29] James, LF; Lijoi, A.; Prünster, I., Conjugacy as a distinctive feature of the Dirichlet process, Scand. J. Stat., 33, 1, 105-120 (2006) · Zbl 1121.62028
[30] Kingman, J.F.C.: Random discrete distribution. J. R. Stat. Soc. Ser. B Stat. Methodol. 37, 1-22 (1975) · Zbl 0331.62019
[31] Lazarsfeld, PF; Henry, NW, Latent structure analysis (1968), Boston, MA: Houghton Mifflin, Boston, MA
[32] Lijoi, A.; Nipoti, B., A class of hazard rate mixtures for combining survival data from different experiments, J. Am. Stat. Assoc., 109, 506, 802-814 (2014) · Zbl 1367.62281
[33] Lijoi, A.; Prünster, I.; Holmes, CC; Muller, P.; Walker, SG, Models beyond the Dirichlet process, Hjort NL (2010), Bayesian Nonparametrics: Cambridge University Press, Bayesian Nonparametrics
[34] Lijoi, A.; Mena, RH; Prünster, I., Hierarchical mixture modeling with normalized inverse-Gaussian priors, J. Am. Stat. Assoc., 100, 472, 1278-1291 (2005) · Zbl 1117.62386
[35] Lijoi, A., Mena, R.H., Prünster, I.: Controlling the reinforcement in Bayesian non-parametric mixture models. J. R. Stat. Soc. Ser. B Stat. Methodol. 69(4), 715-740 (2007)
[36] Lijoi, A.; Nipoti, B.; Prünster, I., Bayesian inference with dependent normalized completely random measures, Bernoulli, 20, 3, 1260-1291 (2014) · Zbl 1309.60048
[37] Lijoi, A.; Nipoti, B.; Prünster, I., Dependent mixture models: clustering and borrowing information, Comput. Stat. Data Anal., 71, 417-433 (2014) · Zbl 06975398
[38] Lijoi, A., Prünster, I., Rigon, T.: Finite-dimensional discrete random structures and Bayesian clustering. Tech. rep. Collegio Carlo Alberto, n. 600 (2019)
[39] Lo, A.Y.: On a class of Bayesian nonparametric estimates: I. density estimates. Ann. Stat. 12(1), 351-357 (1984) · Zbl 0557.62036
[40] MacEachern, SN.: Dependent nonparametric processes. In: ASA Proceedings of the Section on Bayesian Statistical Science, Alexandria, VA: American Statistical Association, pp. 50-55 (1999)
[41] Muliere, P.; Tardella, L., Approximating distributions of random functionals of Ferguson-Dirichlet priors, Can. J. Stat., 26, 2, 283-297 (1998) · Zbl 0913.62010
[42] Perman, M.: Random discrete distributions derived from subordinators. ProQuest LLC, Ann Arbor, MI, thesis (Ph.D.)-University of California, Berkeley (1990)
[43] Perman, M.; Pitman, J.; Yor, M., Size-biased sampling of Poisson point processes and excursions, Prob. Theory Related Fields, 92, 1, 21-39 (1992) · Zbl 0741.60037
[44] Pitman, J., Some Developments of the Blackwell-Macqueen Urn Scheme, Stat. Prob. Game Theory, 30, 245-267 (1996)
[45] Pitman, J.; Yor, M., The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator, Ann. Prob., 25, 2, 855-900 (1997) · Zbl 0880.60076
[46] Regazzini, E.; Lijoi, A.; Prünster, I., Distributional results for means of normalized random measures with independent increments, Ann. Stat., 31, 2, 560-585 (2003) · Zbl 1068.62034
[47] Roberts, GO; Rosenthal, JS, Examples of adaptive MCMC, J Comput. Graphical Stat., 18, 2, 349-367 (2009)
[48] Sethuraman, J., A constructive definition of Dirichlet priors, Stat. Sin., 4, 2, 639-650 (1994) · Zbl 0823.62007
[49] Stouffer, SA; Toby, J., Role conflict and personality, Am. J. Sociol., 56, 5, 395-406 (1951)
[50] Teh, YW; Jordan, MI; Holmes, CC; Muller, P.; Walker, SG, Hierarchical Bayesian nonparametric models with applications, Hjort NL (2010), Bayesian Nonparametrics: Cambridge University Press, Bayesian Nonparametrics
[51] Teh, YW; Jordan, MI; Beal, MJ; Blei, DM, Hierarchical Dirichlet processes, J. Am. Stat. Assoc., 101, 476, 1-41 (2006)
[52] Zhang, L.; Guindani, M.; Versace, F.; Engelmann, JM; Vannucci, M., A spatiotemporal nonparametric Bayesian model of multi-subject fMRI data, Ann. Appl. Stat., 10, 2, 638-666 (2016) · Zbl 1400.62299
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.