Weakly informative reparameterizations for location-scale mixtures. (English) Zbl 07498995

Summary: While mixtures of Gaussian distributions have been studied for more than a century, the construction of a reference Bayesian analysis of those models remains unsolved, with a general prohibition of improper priors due to the ill-posed nature of such statistical objects. This difficulty is usually bypassed by an empirical Bayes resolution. By creating a new parameterization centered on the mean and possibly the variance of the mixture distribution itself, we manage to develop here a weakly informative prior for a wide class of mixtures with an arbitrary number of components. We demonstrate that some posterior distributions associated with this prior and a minimal sample size are proper. We provide Markov chain Monte Carlo (MCMC) implementations that exhibit the expected exchangeability. We only study here the univariate case, the extension to multivariate location-scale mixtures being currently under study. An R package called Ultimixt is associated with this article. Supplementary material for this article is available online.


62-XX Statistics
Full Text: DOI arXiv


[1] Andrews, D. F.; Mallows, C. L., Scale Mixtures of Normal Distributions,, Journal of the Royal Statistical Society, 36, 99-102 (1974) · Zbl 0282.62017
[2] Benaglia, T.; Chauveau, D.; Hunter, D.; Young, D., mixtools: An R Package for Analyzing Finite Mixture Models, Journal of Statistical Software, 32, 1-29 (2009)
[3] Berger, J., The Case for Objective Bayesian Analysis, Bayesian Analysis, 1, 1-17 (2004)
[4] Berger, J.; Bernardo, J.; Sun, D., Natural Induction: An Objective Bayesian Approach, Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103, 125-135 (2009) · Zbl 1177.62030
[5] Berkhof, J.; van Mechelen, I.; Gelman, A., A Bayesian Approach to the Selection and Testing of Mixture Models, Statistica Sinica, 13, 423-442 (2003) · Zbl 1015.62019
[6] Bernardo, J.; Giròn, F.; Bernardo, J.; DeGroot, M.; Lindley, D.; Smith, A., Bayesian Statistics, 3, A Bayesian Analysis of Simple Mixture Problems,, 67-78 (1988), Oxford: Oxford University Press, Oxford
[7] Celeux, G.; Hurn, M.; Robert, C., Computational and Inferential Difficulties With Mixture Posterior Distributions, Journal of the American Statistical Association, 95, 957-979 (2000) · Zbl 0999.62020
[8] Chib, S., Marginal Likelihood From the Gibbs Output, Journal of the American Statistical Association, 90, 1313-1321 (1995) · Zbl 0868.62027
[9] Diebolt, J.; Robert, C., Estimation des Paramètres d’un Mélange par Échantillonnage Bayésien, Notes aux Comptes-Rendus de l’Académie des Sciences I, 311, 653-658 (1990) · Zbl 0711.62026
[10] ———, Estimation of Finite Mixture Distributions by Bayesian Sampling,, Journal of the Royal Statistical Society, 56, 363-375 (1994) · Zbl 0796.62028
[11] Escobar, M.; West, M., Bayesian Prediction and Density Estimation, Journal of the American Statistical Association, 90, 577-588 (1995) · Zbl 0826.62021
[12] Feller, W., An Introduction to Probability Theory and its Applications, 2 (1971), New York: Wiley, New York · Zbl 0219.60003
[13] Figueiredo, M.; Jain, A., Unsupervised Learning of Finite Mixture Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 381-396 (2002)
[14] Frühwirth-Schnatter, S., Markov Chain Monte Carlo Estimation of Classical and Dynamic Switching and Mixture Models, Journal of the American Statistical Association, 96, 194-209 (2001) · Zbl 1015.62022
[15] ———, Finite Mixture and Markov Switching Models (2006), New York: Springer-Verlag, New York, New York
[16] Gelman, A.; Gilks, W.; Roberts, G.; Berger, J.; Bernardo, J.; Dawid, A.; Lindley, D.; Smith, A., Bayesian Statistics, 5, Efficient Metropolis Jumping Rules,, 599-608 (1996), Oxford: Oxford University Press, Oxford
[17] Gelman, A.; King, G., Estimating the Electoral Consequences of Legislative Redistricting, Journal of the American Statistical Association, 85, 274-282 (1990)
[18] Gelman, A.; Rubin, D., Inference From Iterative Simulation Using Multiple Sequences, Statistical Science, 7, 457-472 (1992) · Zbl 1386.65060
[19] Geweke, J., Interpretation and Inference in Mixture Models: Simple MCMC Works, Computational Statistics and Data Analysis, 51, 3529-3550 (2007) · Zbl 1161.62338
[20] Gleser, L., The Gamma Distribution as a Mixture of Exponential Distributions, American Statistician, 43, 115-117 (1989)
[21] Gleser, M.; Carlin, B. P.; Srivastiva, M. S., Probability Matching Priors for Linear Calibration, TEST, 4, 333-357 (1995) · Zbl 0844.62024
[22] Grazian, C.; Robert, C.; Frühwirth-Schnatter, S.; Bitto, A.; Kastner, G.; Posekany, A., Bayesian Statistics from Methods to Models and Applications, 126, Jeffreys Priors for Mixture Estimation,, 37-48 (2015), New York, NY: Springer, New York, NY
[23] Griffin, J. E., Default Priors for Density Estimation With Mixture Models, Bayesian Analysis, 5, 45-64 (2010) · Zbl 1330.62127
[24] Jasra, A.; Holmes, C.; Stephens, D., Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modeling, Statistical Science, 20, 50-67 (2005) · Zbl 1100.62032
[25] Jeffreys, H., Theory of Probability (1939), Oxford: The Clarendon Press, Oxford · JFM 65.0546.04
[26] Kamary, K.; Lee, K., Ultimixt: Bayesian Analysis of a Non-Informative Parametrisation for Gaussian Mixture Distributions, R package version 2.0 (2017)
[27] Kamary, K.; Lee, K.; Robert, C., Non-Informative Reparameterisations for Location-Scale Mixtures. (2016)
[28] Kass, R.; Wasserman, L., Formal Rules of Selecting Prior Distributions: A Review and Annotated Bibliography, Journal of the American Statistical Association, 91, 343-1370 (1996)
[29] Klugman, S.; Panjer, H. H.; Wilmot, G. E., Loss Models, From Data to Decisions (2004), New York: Wiley-Interscience, Wiley, New York · Zbl 1141.62343
[30] Lee, J., Bayesian Hybrid Algorithms and Models: Implementation and Associated Issues, (2009)
[31] Lee, K.; Marin, J.-M.; Mengersen, K.; Robert, C.; Sastry, N. N.; Delampady, M.; Rajeev, B., Perspectives in Mathematical Sciences I: Probability and Statistics, Bayesian Inference on Mixtures of Distributions,, 165-202 (2009), Singapore: World Scientific, Singapore
[32] Marin, J.-M.; Mengersen, K.; Robert, C.; Rao, C.; Dey, D., Handbook of Statistics, 25, Bayesian Modelling and Inference on Mixtures of Distributions,, 459-507 (2005), New York: Springer-Verlag, New York
[33] McGrory, C. A.; Alston, C. L.; Mengersen, K. L.; Pettitt, A. N., Case Studies in Bayesian Statistical Modelling and Analysis, Variational Bayesian Inference for Mixture Models,, 388-402 (2013), New York, NY: Wiley, New York, NY
[34] Mengersen, K.; Robert, C.; Berger, J.; Bernardo, J.; Dawid, A.; Lindley, D.; Smith, A., Bayesian Statistics, 5, Testing for Mixtures: A Bayesian Entropic Approach (with discussion),, 255-276 (1996), Oxford: Oxford University Press, Oxford
[35] Neal, R., Erroneous Results in “Marginal Likelihood From the Gibbs Output, (1999)
[36] Richardson, S.; Green, P., On Bayesian Analysis of Mixtures With an Unknown Number of Components, Journal of the Royal Statistical Society, 59, 731-792 (1997) · Zbl 0891.62020
[37] Rissanen, J., Optimal Estimation of Parameters (2012), Cambridge: Cambridge University Press, Cambridge · Zbl 1292.62016
[38] Robert, C.; Titterington, M., Reparameterisation Strategies for Hidden Markov Models and Bayesian Approaches to Maximum Likelihood Estimation, Statistics and Computing, 8, 145-158 (1998)
[39] Roberts, G. O.; Gelman, A.; Gilks, W. R., Weak Convergence and Optimal Scaling of Random Walk Metropolis Algorithms, The Annals of Applied Probability, 7, 110-120 (1997) · Zbl 0876.60015
[40] Roberts, G. O.; Rosenthal, S. J., Optimal Scaling for Various Metropolis-Hastings Algorithms, Statistical Science, 16, 351-367 (2001) · Zbl 1127.65305
[41] ———, Examples of Adaptive MCMC,, Journal of Computational and Graphical Statistics, 18, 349-367 (2009)
[42] Roeder, K., Density Estimation With Confidence Sets Exemplified by Superclusters and Voids in Galaxies, Journal of the American Statistical Association, 85, 617-624 (1990) · Zbl 0704.62103
[43] Rossi, P.; McCulloch, R., Bayesm: Bayesian Inference for Marketing/Micro-Econometrics, R Package Version, 3.0-2 (2010)
[44] Rousseau, J.; Mengersen, K., Asymptotic Behaviour of the Posterior Distribution in Overfitted Mixture Models,, Journal of the Royal Statistical Society, 73, 689-710 (2011) · Zbl 1228.62034
[45] Rubio, F.; Steel, M., Inference in Two-Piece Location-Scale Models With Jeffreys Priors, Bayesian Analysis, 9, 1-22 (2014) · Zbl 1327.62157
[46] Stephens, M., Bayesian Analysis of Mixture Models With an Unknown Number of Components—An Alternative to Reversible Jump Methods, Annals of Statistics, 28, 40-74 (2000) · Zbl 1106.62316
[47] ———, Dealing With Label Switching in Mixture Models,, Journal of the Royal Statistical Society, 62, 795-809 (2000) · Zbl 0957.62020
[48] Tanner, M.; Wong, W., The Calculation of Posterior Distributions by Data Augmentation, Journal of the American Statistical Association, 82, 528-550 (1987) · Zbl 0619.62029
[49] Thompson, J.; Kinghorn, B., CATMAN-A Program to Measure CAT-Scans for Prediction of Body Components in Live Animals, Proceeding of the Australian Association of Animal Breeding and Genetics, 10, 560-564 (1992)
[50] Wasserman, L., Asymptotic Inference for Mixture Models by Using Data-Dependent Priors,, Journal of the Royal Statistical Society, 61, 159-180 (1999) · Zbl 0976.62028
[51] Welch, B.; Peers, H., On Formulae for Confidence Points Based on Integrals of Weighted Likelihoods,, Journal of the Royal Statistical Society, 25, 318-329 (1963) · Zbl 0117.14205
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.