Weakly informative reparameterizations for location-scale mixtures. (English) Zbl 07498995

Kamary, Kaniav; Lee, Jeong Eun; Robert, Christian P.

J. Comput. Graph. Stat. 27, No. 4, 836-848 (2018).

Summary: While mixtures of Gaussian distributions have been studied for more than a century, the construction of a reference Bayesian analysis of those models remains unsolved, with a general prohibition of improper priors due to the ill-posed nature of such statistical objects. This difficulty is usually bypassed by an empirical Bayes resolution. By creating a new parameterization centered on the mean and possibly the variance of the mixture distribution itself, we manage to develop here a weakly informative prior for a wide class of mixtures with an arbitrary number of components. We demonstrate that some posterior distributions associated with this prior and a minimal sample size are proper. We provide Markov chain Monte Carlo (MCMC) implementations that exhibit the expected exchangeability. We only study here the univariate case, the extension to multivariate location-scale mixtures being currently under study. An R package called Ultimixt is associated with this article. Supplementary material for this article is available online.

MSC:

62-XX

Statistics

Keywords:

Bayesian analysis; compound distributions; Dirichlet prior; exchangeability; improper prior; noninformative prior

Software:

Ultimixt; CATMAN; bayesm; mixtools

References:

[1]	Andrews, D. F.; Mallows, C. L., Scale Mixtures of Normal Distributions,, Journal of the Royal Statistical Society, 36, 99-102 (1974) · Zbl 0282.62017
[2]	Benaglia, T.; Chauveau, D.; Hunter, D.; Young, D., mixtools: An R Package for Analyzing Finite Mixture Models, Journal of Statistical Software, 32, 1-29 (2009)
[3]	Berger, J., The Case for Objective Bayesian Analysis, Bayesian Analysis, 1, 1-17 (2004)
[4]	Berger, J.; Bernardo, J.; Sun, D., Natural Induction: An Objective Bayesian Approach, Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103, 125-135 (2009) · Zbl 1177.62030
[5]	Berkhof, J.; van Mechelen, I.; Gelman, A., A Bayesian Approach to the Selection and Testing of Mixture Models, Statistica Sinica, 13, 423-442 (2003) · Zbl 1015.62019
[6]	Bernardo, J.; Giròn, F.; Bernardo, J.; DeGroot, M.; Lindley, D.; Smith, A., Bayesian Statistics, 3, A Bayesian Analysis of Simple Mixture Problems,, 67-78 (1988), Oxford: Oxford University Press, Oxford
[7]	Celeux, G.; Hurn, M.; Robert, C., Computational and Inferential Difficulties With Mixture Posterior Distributions, Journal of the American Statistical Association, 95, 957-979 (2000) · Zbl 0999.62020
[8]	Chib, S., Marginal Likelihood From the Gibbs Output, Journal of the American Statistical Association, 90, 1313-1321 (1995) · Zbl 0868.62027
[9]	Diebolt, J.; Robert, C., Estimation des Paramètres d’un Mélange par Échantillonnage Bayésien, Notes aux Comptes-Rendus de l’Académie des Sciences I, 311, 653-658 (1990) · Zbl 0711.62026
[10]	———, Estimation of Finite Mixture Distributions by Bayesian Sampling,, Journal of the Royal Statistical Society, 56, 363-375 (1994) · Zbl 0796.62028
[11]	Escobar, M.; West, M., Bayesian Prediction and Density Estimation, Journal of the American Statistical Association, 90, 577-588 (1995) · Zbl 0826.62021
[12]	Feller, W., An Introduction to Probability Theory and its Applications, 2 (1971), New York: Wiley, New York · Zbl 0219.60003
[13]	Figueiredo, M.; Jain, A., Unsupervised Learning of Finite Mixture Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 381-396 (2002)
[14]	Frühwirth-Schnatter, S., Markov Chain Monte Carlo Estimation of Classical and Dynamic Switching and Mixture Models, Journal of the American Statistical Association, 96, 194-209 (2001) · Zbl 1015.62022
[15]	———, Finite Mixture and Markov Switching Models (2006), New York: Springer-Verlag, New York, New York
[16]	Gelman, A.; Gilks, W.; Roberts, G.; Berger, J.; Bernardo, J.; Dawid, A.; Lindley, D.; Smith, A., Bayesian Statistics, 5, Efficient Metropolis Jumping Rules,, 599-608 (1996), Oxford: Oxford University Press, Oxford
[17]	Gelman, A.; King, G., Estimating the Electoral Consequences of Legislative Redistricting, Journal of the American Statistical Association, 85, 274-282 (1990)
[18]	Gelman, A.; Rubin, D., Inference From Iterative Simulation Using Multiple Sequences, Statistical Science, 7, 457-472 (1992) · Zbl 1386.65060
[19]	Geweke, J., Interpretation and Inference in Mixture Models: Simple MCMC Works, Computational Statistics and Data Analysis, 51, 3529-3550 (2007) · Zbl 1161.62338
[20]	Gleser, L., The Gamma Distribution as a Mixture of Exponential Distributions, American Statistician, 43, 115-117 (1989)
[21]	Gleser, M.; Carlin, B. P.; Srivastiva, M. S., Probability Matching Priors for Linear Calibration, TEST, 4, 333-357 (1995) · Zbl 0844.62024
[22]	Grazian, C.; Robert, C.; Frühwirth-Schnatter, S.; Bitto, A.; Kastner, G.; Posekany, A., Bayesian Statistics from Methods to Models and Applications, 126, Jeffreys Priors for Mixture Estimation,, 37-48 (2015), New York, NY: Springer, New York, NY
[23]	Griffin, J. E., Default Priors for Density Estimation With Mixture Models, Bayesian Analysis, 5, 45-64 (2010) · Zbl 1330.62127
[24]	Jasra, A.; Holmes, C.; Stephens, D., Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modeling, Statistical Science, 20, 50-67 (2005) · Zbl 1100.62032
[25]	Jeffreys, H., Theory of Probability (1939), Oxford: The Clarendon Press, Oxford · JFM 65.0546.04
[26]	Kamary, K.; Lee, K., Ultimixt: Bayesian Analysis of a Non-Informative Parametrisation for Gaussian Mixture Distributions, R package version 2.0 (2017)
[27]	Kamary, K.; Lee, K.; Robert, C., Non-Informative Reparameterisations for Location-Scale Mixtures. (2016)
[28]	Kass, R.; Wasserman, L., Formal Rules of Selecting Prior Distributions: A Review and Annotated Bibliography, Journal of the American Statistical Association, 91, 343-1370 (1996)
[29]	Klugman, S.; Panjer, H. H.; Wilmot, G. E., Loss Models, From Data to Decisions (2004), New York: Wiley-Interscience, Wiley, New York · Zbl 1141.62343
[30]	Lee, J., Bayesian Hybrid Algorithms and Models: Implementation and Associated Issues, (2009)
[31]	Lee, K.; Marin, J.-M.; Mengersen, K.; Robert, C.; Sastry, N. N.; Delampady, M.; Rajeev, B., Perspectives in Mathematical Sciences I: Probability and Statistics, Bayesian Inference on Mixtures of Distributions,, 165-202 (2009), Singapore: World Scientific, Singapore
[32]	Marin, J.-M.; Mengersen, K.; Robert, C.; Rao, C.; Dey, D., Handbook of Statistics, 25, Bayesian Modelling and Inference on Mixtures of Distributions,, 459-507 (2005), New York: Springer-Verlag, New York
[33]	McGrory, C. A.; Alston, C. L.; Mengersen, K. L.; Pettitt, A. N., Case Studies in Bayesian Statistical Modelling and Analysis, Variational Bayesian Inference for Mixture Models,, 388-402 (2013), New York, NY: Wiley, New York, NY
[34]	Mengersen, K.; Robert, C.; Berger, J.; Bernardo, J.; Dawid, A.; Lindley, D.; Smith, A., Bayesian Statistics, 5, Testing for Mixtures: A Bayesian Entropic Approach (with discussion),, 255-276 (1996), Oxford: Oxford University Press, Oxford
[35]	Neal, R., Erroneous Results in “Marginal Likelihood From the Gibbs Output, (1999)
[36]	Richardson, S.; Green, P., On Bayesian Analysis of Mixtures With an Unknown Number of Components, Journal of the Royal Statistical Society, 59, 731-792 (1997) · Zbl 0891.62020
[37]	Rissanen, J., Optimal Estimation of Parameters (2012), Cambridge: Cambridge University Press, Cambridge · Zbl 1292.62016
[38]	Robert, C.; Titterington, M., Reparameterisation Strategies for Hidden Markov Models and Bayesian Approaches to Maximum Likelihood Estimation, Statistics and Computing, 8, 145-158 (1998)
[39]	Roberts, G. O.; Gelman, A.; Gilks, W. R., Weak Convergence and Optimal Scaling of Random Walk Metropolis Algorithms, The Annals of Applied Probability, 7, 110-120 (1997) · Zbl 0876.60015
[40]	Roberts, G. O.; Rosenthal, S. J., Optimal Scaling for Various Metropolis-Hastings Algorithms, Statistical Science, 16, 351-367 (2001) · Zbl 1127.65305
[41]	———, Examples of Adaptive MCMC,, Journal of Computational and Graphical Statistics, 18, 349-367 (2009)
[42]	Roeder, K., Density Estimation With Confidence Sets Exemplified by Superclusters and Voids in Galaxies, Journal of the American Statistical Association, 85, 617-624 (1990) · Zbl 0704.62103
[43]	Rossi, P.; McCulloch, R., Bayesm: Bayesian Inference for Marketing/Micro-Econometrics, R Package Version, 3.0-2 (2010)
[44]	Rousseau, J.; Mengersen, K., Asymptotic Behaviour of the Posterior Distribution in Overfitted Mixture Models,, Journal of the Royal Statistical Society, 73, 689-710 (2011) · Zbl 1228.62034
[45]	Rubio, F.; Steel, M., Inference in Two-Piece Location-Scale Models With Jeffreys Priors, Bayesian Analysis, 9, 1-22 (2014) · Zbl 1327.62157
[46]	Stephens, M., Bayesian Analysis of Mixture Models With an Unknown Number of Components—An Alternative to Reversible Jump Methods, Annals of Statistics, 28, 40-74 (2000) · Zbl 1106.62316
[47]	———, Dealing With Label Switching in Mixture Models,, Journal of the Royal Statistical Society, 62, 795-809 (2000) · Zbl 0957.62020
[48]	Tanner, M.; Wong, W., The Calculation of Posterior Distributions by Data Augmentation, Journal of the American Statistical Association, 82, 528-550 (1987) · Zbl 0619.62029
[49]	Thompson, J.; Kinghorn, B., CATMAN-A Program to Measure CAT-Scans for Prediction of Body Components in Live Animals, Proceeding of the Australian Association of Animal Breeding and Genetics, 10, 560-564 (1992)
[50]	Wasserman, L., Asymptotic Inference for Mixture Models by Using Data-Dependent Priors,, Journal of the Royal Statistical Society, 61, 159-180 (1999) · Zbl 0976.62028
[51]	Welch, B.; Peers, H., On Formulae for Confidence Points Based on Integrals of Weighted Likelihoods,, Journal of the Royal Statistical Society, 25, 318-329 (1963) · Zbl 0117.14205