×

Predictive construction of priors in Bayesian nonparametrics. (English) Zbl 1319.62075

Summary: The characterization of models and priors through a predictive approach is a fundamental problem in Bayesian statistics. In the last decades, it has received renewed interest, as the basis of important developments in Bayesian nonparametrics and in machine learning. In this paper, we review classical and recent work based on the predictive approach in these areas. Our focus is on the predictive construction of priors for Bayesian nonparametric inference, for exchangeable and partially exchangeable sequences. Some results are revisited to shed light on theoretical connections among them.

MSC:

62G05 Nonparametric estimation
62F15 Bayesian inference
62-02 Research exposition (monographs, survey articles) pertaining to statistics
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Aldous, D. J. (1981). Representations for partially exchangeable arrays of random variables. Journal of Multivariate Analysis 11 , 581-598. · Zbl 0474.60044 · doi:10.1016/0047-259X(81)90099-3
[2] Antoniak, C. E. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. The Annals of Statistics 2 , 1152-1174. · Zbl 0335.60034 · doi:10.1214/aos/1176342871
[3] Arellano-Valle, R. B. and Bolfarine, H. (1995). On some characterizations of the \(t\)-distribution. Statistics and Probability Letters 25 , 79-85. · Zbl 0838.62040 · doi:10.1016/0167-7152(94)00208-P
[4] Arellano-Valle, R. B., Bolfarine, H. and Iglesias, P. L. (1994). A predictivistic interpretation to the multivariate \(t\)-distribution. Test 3 , 221-236. · Zbl 0813.62002 · doi:10.1007/BF02562703
[5] Bacallado, S. (2011). Bayesian analysis of variable-order, reversible Markov chains. The Annals of Statistics 39 , 838-864. · Zbl 1215.62083 · doi:10.1214/10-AOS857
[6] Barrientos, A. F., Jara, A. and Quintana, F. (2011). On the support of MacEachern’s dependent Dirichlet processes. Technical report, Dept. Statistics, Pontificia Universidad Católica de Chile. · Zbl 1330.60067
[7] Beal, M. J., Ghahramani, Z. and Rasmussen, C. E. (2002). The infinite hidden Markov model. In Advances in Neural Information Processing Systems 14 , 577-584. Cambridge, MA: MIT Press.
[8] Berti, P. and Rigo, P. (1997). A Glivenko-Cantelli theorem for exchangeable random variables. Statistics and Probability Letters 32 , 385-391. · Zbl 0904.60025 · doi:10.1016/S0167-7152(96)00098-3
[9] Blackwell, D. and MacQueen, J. B. (1973). Ferguson distributions via Pólya urn schemes. The Annals of Statistics 1 , 353-355. · Zbl 0276.62010 · doi:10.1214/aos/1176342372
[10] Bulla, P., Muliere, P. and Walker, S. (2009). A Bayesian nonparametric estimator of a multivariate survival function. Journal of Statistical Planning and Inference 139 , 3639-3648. · Zbl 1167.62391 · doi:10.1016/j.jspi.2009.04.012
[11] Caron, F., Davy, M. and Doucet, A. (2007). Generalized Pólya urn for time-varying Dirichlet process mixtures. In 23rd Conference on Uncertainty in Artificial Intelligence (UAI’2007) , Canada.
[12] Cifarelli, D. M. and Regazzini, E. (1978). Problemi statistici nonparametrici in condizioni di scambiabilità parziale e impiego di medie associative. Quaderni Istituto di Matematica Finanziaria , Università di Torino Ser. III 12 , 1-36. English translation available at .
[13] Cifarelli, D. M. and Regazzini, E. (1996). De Finetti’s contribution to probability and statistics. Statistical Science 11 , 253-282. · Zbl 0955.01552 · doi:10.1214/ss/1032280303
[14] Cirillo, P., Hüsler, J. and Muliere, P. (2010). A nonparametric urn-based approach to interacting failing systems with an application to credit risk modeling. International Journal of Theoretical and Applied Finance 13 , 1223-1240. · Zbl 1205.91164 · doi:10.1142/S0219024910006170
[15] Dawid, A. P. (1978). Extendebility of spherical matrix distributions. Journal of Multivariate Analysis 8 , 559-566. · Zbl 0398.62009 · doi:10.1016/0047-259X(78)90033-7
[16] de Finetti, B. (1937). La prévision: Ses lois logiques, ses sources subjectives. Annales de l’Institut Henri Poincaré 7 , 1-68. English translation: Foresight, its logical laws, its subjective sources. In Studies in Subjective Probability (H. E. Kyburg and H. E. Smokler, eds.) (1964) 97-156. New York: Wiley.
[17] de Finetti, B. (1959). La probabilità e la statistica nei rapporti con l’induzione, secondo i diversi punti di vista. In Atti corso CIME su Induzione e Statistica Varenna 1-115. Roma: Cremonese. English translation: Probability, Induction and Statistics (1972) 147-227. New York: Wiley. · Zbl 0124.34801
[18] Diaconis, P., Eaton, M. L. and Lauritzen, S. L. (1992). Finite De Finetti theorems in linear models and multivariate analysis. Scandinavian Journal of Statistics 19 , 289-315. · Zbl 0795.62049
[19] Diaconis, P. and Freedman, D. (1980). de Finetti’s theorem for Markov chains. The Annals of Probability 8 , 115-130. · Zbl 0426.60064 · doi:10.1214/aop/1176994828
[20] Diaconis, P. and Freedman, D. (1984). Partial exchangeability and sufficiency. In Proceedings of the Indian Statistical Institute Golden Jubilee International Conference on Statistics. Applications and New Directions (J. K. Gosh and J. Roy, eds.) 205-236. Calcutta: Indian Statistical Instutute.
[21] Diaconis, P. and Rolles, S. W. W. (2006). Bayesian analysis for reversible Markov Chains. The Annals of Statistics 34 , 1270-1292. · Zbl 1118.62085 · doi:10.1214/009053606000000290
[22] Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. The Annals of Statistics 7 , 269-281. · Zbl 0405.62011 · doi:10.1214/aos/1176344611
[23] Doksum, K. A. (1974). Tailfree and neutral random probabilities and their posterior distributions. The Annals of Probability 2 , 183-201. · Zbl 0279.60097 · doi:10.1214/aop/1176996703
[24] Dunson, D. B. (2010). Nonparametric Bayes applications to biostatistics. In Bayesian Nonparametrics: Principles and Practice (N. L. Hjort, C. Holmes, P. Müller and S. G. Walker, eds.). Cambridge, UK: Cambridge Univ. Press.
[25] Eaton, M. L., Fortini, S. and Regazzini, E. (1993). Spherical symmetry: An elementary justification. Journal of the Italian Statistical Society 2 , 1-16. · Zbl 1446.62133
[26] Escobar, M. D. and West, M. (1995). Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association 90 , 577-588. · Zbl 0826.62021 · doi:10.2307/2291069
[27] Ewens, W. J. (1972). The sampling theory of selectively neutral alleles. Theoretical Population Biology 3 , 87-112. · Zbl 0245.92009 · doi:10.1016/0040-5809(72)90035-4
[28] Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. The Annals of Statistics 1 , 209-230. · Zbl 0255.62037 · doi:10.1214/aos/1176342360
[29] Fisher, R. A., Corbet, A. S. and Williams, C. B. (1943). The relation between the number of species and the number of individuals in a random sample of an animal population. Journal of Animal Ecology 12 , 42-58.
[30] Fortini, S., Ladelli, L. and Regazzini, E. (2000). Exchangeability, predictive distributions and parametric models. Sankhya, Ser. A 62 , 86-109. · Zbl 0973.62002
[31] Fortini, S. and Petrone, S. (2011). Hierarchical reinforced urn processes. Technical report, Bocconi Univ., Milano. · Zbl 1282.62018
[32] Fortini, S., Ladelli, L., Petris, G. and Regazzini, E. (2002). On mixtures of distributions of Markov chains. Stochastic Processes and Their Applications 100 , 147-165. · Zbl 1060.60071 · doi:10.1016/S0304-4149(02)00093-5
[33] Freedman, D. (1963). Invariants under mixing that generalize de Finetti’s theorem: Continuous time parameter. The Annals of Mathematical Statistics 34 , 1194-1216. · Zbl 0253.60064 · doi:10.1214/aoms/1177703856
[34] Fox, E. B., Sudderth, E. B., Jordan, M. I. and Willsky, A. S. (2011). A Sticky HDP-HMM with application to speaker diarization. The Annals of Applied Statistics 5 , 1020-1056. · Zbl 1232.62077 · doi:10.1214/10-AOAS395
[35] Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics . New York: Springer-Verlag. · Zbl 1029.62004
[36] Gnedin, A. V. and Pitman, J. (2005). Exchangeable Gibbs partitions and Stirling triangles. Zap. Nauchn. Sem. S.-Peterburg. Otdel. Mat. Inst. Steklov. (POMI) 325 , 83-102. · Zbl 1293.60010
[37] Griffin, J. E. and Steel, M. (2011). Stick-breaking autoregressive processes. Journal of Econometrics 162 , 383-396. · Zbl 1441.62709
[38] Griffiths, T. L. and Ghahramani, Z. (2006). Infinite latent feature models and the Indian Buffet Process. In Advances in Neural Information Processing Systems (NIPS) (Y. Weiss, B. Schölkopf and J. Platt, eds.), 475-482. Cambridge, MA: MIT Press.
[39] Hjort, N. L. (1990). Nonparametric Bayes estimators based on beta process in models for life history data. The Annals of Statistics 18 , 1259-1294. · Zbl 0711.62033 · doi:10.1214/aos/1176347749
[40] Hjort, N. L., Holmes, C., Müller, P. and Walker, S. G. (2010). Bayesian Nonparametrics . Cambridge, UK: Cambridge Univ. Press. · Zbl 1192.62080
[41] Hoppe, F. M. (1984). Pólya-like urns and the Ewens’s sampling formula. Journal Mathematical Biology 20 , 91-94. · Zbl 0547.92009 · doi:10.1007/BF00275863
[42] Iglesias, P. L., Loschi, R. H., Pereira, C. A. B. and Wechsler, S. (2009). A note on extendibility and predictivistic inference in finite populations. Brazilian Journal of Probability and Statistics 23 , 216-226. · Zbl 1298.62043
[43] Ishwaran, H. and James, L. F. (2001). Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association 96 , 161-173. · Zbl 1014.62006 · doi:10.1198/016214501750332758
[44] Kallenberg, O. (2005). Probabilistic Symmetries and Invariance Principles . New York: Springer. · Zbl 1084.60003
[45] Kingman, J. F. C. (1967). Completely random measures. Pacific Journal of Mathematics 21 , 59-78. · Zbl 0155.23503 · doi:10.2140/pjm.1967.21.59
[46] Kingman, J. F. C. (1972). On random sequences with spherical symmetry. Biometrika 59 , 492-493. · Zbl 0238.60025 · doi:10.1093/biomet/59.2.492
[47] Kingman, J. F. C. (1975). Random discrete distributions. Journal of the Royal Statistical Society, Ser. B 37 , 1-22. · Zbl 0331.62019
[48] Kingman, J. F. C. (1978). The representation of partition structures. Journal of the London Mathematical Society 18 , 374-380. · Zbl 0415.92009 · doi:10.1112/jlms/s2-18.2.374
[49] Lijoi, A., Mena, R. H. and Prünster, I. (2005). Hierarchical mixture modelling with normalized inverse Gaussian priors. Journal of the American Statistical Association 100 , 1278-1291. · Zbl 1117.62386 · doi:10.1198/016214505000000132
[50] Lijoi, A. and Prünster, I. (2010). Models beyond the Dirichlet process. In Bayesian Nonparametrics (N. L. Hjort, C. Holmes, P. Müller and S. G. Walker, eds.) 80-136. Cambridge, UK: Cambridge Univ. Press.
[51] Loschi, R. H., Iglesias, P. I. and Arellano-Valle, R. B. (2003). Predictivistic characterizations of multivariate student-\(t\) models. Journal of Multivariate Analysis 85 , 10-23. · Zbl 1013.62061 · doi:10.1016/S0047-259X(02)00034-9
[52] MacEachern, S. N. (1994). Estimating normal means with a conjugate style Dirichlet process prior. Communications in Statistics: Simulation and Computation 23 , 727-741. · Zbl 0825.62053 · doi:10.1080/03610919408813196
[53] MacEachern, S. N. (1999). Dependent nonparametric processes. In ASA Proceedings of the Section on Bayesian Statistical Science 50-55. Alexandria, VA: American Statistical Association.
[54] MacEachern, S. N. (2001). Decision theoretic aspects of dependent nonparametric processes. In Bayesian Methods with Applications to Science, Policy, and Official Statistics (E. George, ed.) 551-560. Crete: International Society for Bayesian Analysis.
[55] McCloskey, J. W. (1965). A model for the distribution of individuals by species in an environment. Ph.D. thesis, Michigan State Univ.
[56] Muliere, P. and Petrone, S. (1993). A Bayesian predictive approach to sequential search for an optimal dose: Parametric and nonparametric models. Journal of the Italian Statistical Society 2 , 349-364. · Zbl 1446.62283
[57] Muliere, P., Secchi, P. and Walker, S. G. (2000). Urn schemes and reinforced random walks. Stochastic Processes and Their Applications 88 , 59-78. · Zbl 1045.62006 · doi:10.1016/S0304-4149(99)00119-2
[58] Müller, P. and Quintana, F. A. (2004). Nonparametric Bayesian data analysis. Statistical Science 19 , 95-110. · Zbl 1057.62032 · doi:10.1214/088342304000000017
[59] Perman, M., Pitman, J. and Yor, M. (1992). Size-biased sampling of Poisson point processes and excursions. Probability Theory and Related Fields 92 , 21-39. · Zbl 0741.60037 · doi:10.1007/BF01205234
[60] Petrone, S., Guindani, M. and Gelfand, A. E. (2009). Hybrid Dirichlet mixture models for functional data. Journal of the Royal Statistical Society, Ser. B 71 , 755-782. · Zbl 1248.62079
[61] Petrone, S. and Raftery, A. E. (1997). A note on the Dirichlet process prior in Bayesian nonparametric inference with partial exchangeability. Statistics and Probability Letters 36 , 69-83. · Zbl 0953.62105 · doi:10.1016/S0167-7152(97)00050-3
[62] Petrone, S. and Veronese, P. (2010). Feller operators and mixture priors in Bayesian nonparametrics. Statistica Sinica 20 , 379-404. · Zbl 1191.62048
[63] Pitman, J. (1995). Exchangeable and partially exchangeable random partitions. Probability Theory and Related Fields 102 , 145-158. · Zbl 0821.60047 · doi:10.1007/BF01213386
[64] Pitman, J. (1996). Some developments of the Blackwell-MacQueen urn scheme. In Statistics, Probability and Game Theory (T. S. Ferguson et al., eds.). Lecture Notes-Monograph Series 30 , 245-267. Hayward, CA: IMS.
[65] Pitman, J. (2003). Poisson-Kingman partitions. In: Statistics and Science: A Festschrift for Terry Speed (D. R. Goldstein, ed.). IMS Lecture Notes-Monograph Series 40 , 1-34. Beachwood: IMS.
[66] Pitman, J. and Yor, M. (1997). The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. The Annals of Probability 25 , 855-900. · Zbl 0880.60076 · doi:10.1214/aop/1024404422
[67] Quintana, F. A. and Iglesias, P. L. (2003). Bayesian clustering and product partition models. Journal of the Royal Statistical Society, Ser. B 65 , 557-574. · Zbl 1065.62115 · doi:10.1111/1467-9868.00402
[68] Regazzini, E., Lijoi, A. and Prünster, I. (2003). Distributional results for means of normalized random measures with independent increments. The Annals of Statistics 31 , 560-585. · Zbl 1068.62034 · doi:10.1214/aos/1051027881
[69] Roy, D. M. and Teh, Y. W. (2009). The Mondrian Process. In Advances in Neural Information Processing Systems (NIPS) (D. Koller, Y. Bengio, D. Schuurmans, L. Bottou and A. Culotta, eds.) 21 , 1377-1384. NIPS.
[70] Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statistica Sinica 4 , 639-650. · Zbl 0823.62007
[71] Smith, A. F. M. (1981). On random sequences with centered spherical symmetry. Journal of the Royal Statistics Society, Ser. B 43 , 208-209. · Zbl 0471.62016
[72] Teh, Y. W. and Jordan, M. I. (2010). Hierarchical Bayesian nonparametric models with applications In Bayesian Nonparametrics (N. L. Hjort, C. Holmes, P. Müller and S. G. Walker, eds.) 158-207. Cambridge, UK: Cambridge Univ. Press.
[73] Teh, Y. W., Jordan, M. I., Beal, M. J. and Blei, D. M. (2006). Hierarchical Dirichlet processes. Journal of the American Statistical Association 101 , 1566-1581. · Zbl 1171.62349 · doi:10.1198/016214506000000302
[74] Thibaux, R. and Jordan, M. I. (2007). Hierarchical beta processes and the Indian buffet process. In Journal of Machine Learning Research-Proceedings Track 2 564-571. Proceedings of AISTATS, San Juan, Puerto Rico, 2007.
[75] Walker, S. and Muliere, P. (1997). Beta-Stacy processes and a generalization of the Pólya-urn scheme. The Annals of Statistics 25 , 1762-1780. · Zbl 0928.62067 · doi:10.1214/aos/1031594741
[76] Walker, S. G. and Muliere, P. (2003). A bivariate Dirichlet process. Statistics and Probability Letters 64 , 1-7. · Zbl 1113.62313 · doi:10.1016/S0167-7152(03)00124-X
[77] Zabell, S. L. (1982). W. E. Johnson’s “sufficientness” postulate. The Annals of Statistics 10 , 1090-1099. · Zbl 0512.62007 · doi:10.1214/aos/1176345975
[78] Zabell, S. L. (1995). Characterizing Markov exchangeable sequences. Journal of Theoretical Probability 8 , 175-178. · Zbl 0814.60029 · doi:10.1007/BF02213460
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.