Generalized smooth finite mixtures. (English) Zbl 1443.62085

Summary: We propose a general class of models and a unified Bayesian inference methodology for flexibly estimating the density of a response variable conditional on a possibly high-dimensional set of covariates. Our model is a finite mixture of component models with covariate-dependent mixing weights. The component densities can belong to any parametric family, with each model parameter being a deterministic function of covariates through a link function. Our MCMC methodology allows for Bayesian variable selection among the covariates in the mixture components and in the mixing weights. The model’s parameterization and variable selection prior are chosen to prevent overfitting. We use simulated and real data sets to illustrate the methodology.


62G07 Density estimation
62F15 Bayesian inference
62P20 Applications of statistics to economics
Full Text: DOI Link


[1] Bastos, J., Ramalho, J., 2010 Nonparametric models of financial leverage decisions, cemapre working paper series, 1005, CEMAPRE Working Papers, available at http://ideas.repec.org/p/cma/wpaper/1005.html.
[2] Casarin, R., Dalla Valle, L., Leisen, F., 2010 Bayesian model selection for beta autoregressive processes. Arxiv preprint arXiv:1008.0121. · Zbl 1330.62113
[3] Chung, Y.; Dunson, D., Nonparametric Bayes conditional distribution modeling with variable selection, Journal of American statistical association, 104, 488, 1646-1660, (2009) · Zbl 1205.62039
[4] Consul, P.; Jain, G., A generalization of the Poisson distribution, Technometrics, 15, 4, 791-799, (1973) · Zbl 0271.60020
[5] Cook, D.; Kieschnick, R.; McCullough, B., Regression analysis of proportions in finance with self selection, Journal of empirical finance, 15, 5, 860-867, (2008)
[6] Czado, C.; Erhardt, V.; Min, A.; Wagner, S., Zero-inflated generalized Poisson models with regression effects on the Mean, dispersion and zero-inflation level applied to patent outsourcing rates, Statistical modelling, 7, 2, 125, (2007)
[7] Denison, D.; Holmes, C.C.; Mallick, B.K.; Smith, A.F.M., Bayesian methods for nonlinear classification and regression, (2002), Wiley
[8] DeSarbo, W.; Cron, W., A maximum likelihood methodology for clusterwise linear regression, Journal of classification, 5, 2, 249-282, (1988) · Zbl 0692.62052
[9] Dunson, D.; Pillai, N.; Park, J., Bayesian density regression, Journal of royal statistical society: series B (statistical methodology), 69, 2, 163-183, (2007) · Zbl 1120.62025
[10] Escobar, M.; West, M., Bayesian density estimation and inference using mixtures, Journal of American statistical association, 90, 430, (1995) · Zbl 0826.62021
[11] Famoye, F.; Singh, K., Zero-inflated generalized Poisson regression model with an application to domestic violence data, Journal of data science, 4, 117-130, (2006)
[12] Fruhwirth-Schnatter, S., Finite mixture and Markov switching models, (2006), Springer-Verlag · Zbl 1108.62002
[13] Gamerman, D., Sampling from the posterior distribution in generalized linear mixed models, Statistics and computing, 7, 1, 57-68, (1997)
[14] Geweke, J., Using simulation methods for Bayesian econometric models: inference, development, and communication, Econometric reviews, 18, 1, 1-73, (1999) · Zbl 0930.62105
[15] Geweke, J.; Keane, M., Smoothly mixing regressions, Journal of econometrics, 138, 1, 252-290, (2007) · Zbl 1418.62455
[16] Giordani, P., Jacobson, T., von Schedvin, E., Villani, M., 2011. Taking the twists into account: Predicting firm bankruptcy risk with splines of financial ratios, Sveriges Riksbank working paper series, 256, Manuscript, available at http://www.riksbank.se.
[17] Green, P.; Richardson, S., Modelling heterogeneity with and without the Dirichlet process, Scandinavian journal of statistics, 28, 2, 355-375, (2001) · Zbl 0973.62031
[18] Hastie, T.; Tibshirani, R., Generalized additive models, (1990), Chapman & Hall/CRC · Zbl 0747.62061
[19] Hilbe, J., Negative binomial regression, (2007), Cambridge University Press NY · Zbl 1131.62068
[20] Jacobs, R.; Jordan, M.; Nowlan, S.; Hinton, G., Adaptive mixtures of local experts, Neural computation, 3, 1, 79-87, (1991)
[21] Jiang, W.; Tanner, M., Hierarchical mixtures-of-experts for exponential family regression models: approximation and maximum likelihood estimation, Annals of statistics, 987-1011, (1999) · Zbl 0957.62032
[22] Jordan, M.; Jacobs, R., Hierarchical mixtures of experts and the EM algorithm, Neural computation, 2, 181-214, (1994)
[23] Kass, R., Bayes factors in practice, The Statistician, 42, 5, 551-560, (1993)
[24] Kohn, R.; Smith, M.; Chan, D., Nonparametric regression using linear combinations of basis functions, Statistics and computing, 11, 4, 313-322, (2001)
[25] Li, F.; Villani, M.; Kohn, R., Flexible modeling of conditional distributions using smooth mixtures of asymmetric student t densities, Journal of statistical planning and inference, 140, 3638-3654, (2010) · Zbl 1233.62076
[26] Muller, P.; Erkanli, A.; West, M., Bayesian curve Fitting using multivariate normal mixtures, Biometrika, 83, 1, 67, (1996) · Zbl 0865.62029
[27] Norets, A., Approximation of conditional densities by smooth mixtures of regressions, The annals of statistics, 38, 3, 1733-1766, (2010) · Zbl 1189.62060
[28] Norets, A., Pelenis, J., 2011 Posterior consistency in conditional density estimation by covariate dependent mixtures. Unpublished manuscript, Princeton University. Available at, http://www.princeton.edu/anorets/consmixreg.pdf. · Zbl 1296.62083
[29] Nott, D.; Kohn, J., Adaptive sampling for Bayesian variable selection, Biometrika, 92, 747-763, (2005) · Zbl 1160.62312
[30] Nott, D.; Leonte, D., Sampling schemes for Bayesian variable selection in generalized linear models, Journal of computational and graphical statistics, 13, 2, 362-382, (2004)
[31] Ntzoufras, I.; Dellaportas, P.; Forster, J., Bayesian variable and link determination for generalised linear models, Journal of statistical planning and inference, 111, 1-2, 165-180, (2003) · Zbl 1033.62026
[32] Peng, F.; Jacobs, R.; Tanner, M., Bayesian inference in mixtures-of-experts and hierarchical mixtures-of-experts models with an application to speech recognition, Journal of American statistical association, 91, 435, (1996) · Zbl 0882.62022
[33] Qi, Y., Minka, T., 2002 Hessian-based Markov chain monte-carlo algorithms. Unpublished manuscript available at web.media.mit.edu/yuanqi/papers/qi-hmh-amit-02.ps.
[34] Rajan, R.; Zingales, L., What do we know about capital structure? some evidence from international data, Journal of finance, 50, 5, 1421-1460, (1995)
[35] Ramalho, E.; Ramalho, J.; Murteira, J., Alternative estimating and testing empirical strategies for fractional regression models, Journal of economic surveys, 25, 19-68, (2009)
[36] Richardson, S.; Green, P., On Bayesian analysis of mixtures with an unknown number of components (with discussion), Journal of royal statistical society: series B (statistical methodology), 59, 4, 731-792, (1997) · Zbl 0891.62020
[37] Riphahn, R.; Wambach, A.; Million, A., Incentive effects in the demand for health care: a bivariate panel count data estimation, Journal of applied econometrics, 18, 4, 387-405, (2003)
[38] Shahbaba, B.; Neal, R., Nonlinear models using Dirichlet process mixtures, Journal of machine learning research, 10, 1829-1850, (2009) · Zbl 1235.62069
[39] Smyth, G., Generalized linear models with varying dispersion, Journal of royal statistical society. series B (methodological), 47-60, (1989)
[40] Stephens, M., Bayesian analysis of mixture models with an unknown number of components-an alternative to reversible jump methods, Annals of statistics, 28, 1, 40-74, (2000) · Zbl 1106.62316
[41] Villani, M.; Kohn, R.; Giordani, P., Regression density estimation using smooth adaptive Gaussian mixtures, Journal of econometrics, 153, 2, 155-173, (2009) · Zbl 1431.62093
[42] Wood, S.; Jiang, W.; Tanner, M., Bayesian mixture of splines for spatially adaptive nonparametric regression, Biometrika, 89, 3, 513, (2002) · Zbl 1136.62340
[43] Wood, S.; Kohn, R.; Cottet, R.; Jiang, W.; Tanner, M., Locally adaptive nonparametric binary regression, Journal of computational and graphical statistics, 17, 2, 352-372, (2008)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.