×

Non-parametric Bayesian inference for continuous density hidden Markov mixture model. (English) Zbl 1487.62099

Summary: In this paper, we present a non-parametric continuous density Hidden Markov mixture model (CDHMMix model) with unknown number of mixtures for blind segmentation or clustering of sequences. In our presented model, the emission distributions of HMMs are chosen to be Gaussian with full, diagonal, or tridiagonal covariance matrices. We apply a Bayesian approach to train our presented model and drive the inference of our model using the Monte Carlo Markov Chain (MCMC) method. For the multivariate Gaussian emission a method that maintains the tridiagonal structure of the covariance is introduced. Moreover, we present a new sampling method for hidden state sequences of HMMs based on the Viterbi algorithm that increases the mixing rate.

MSC:

62M05 Markov processes: estimation; hidden Markov models
62G07 Density estimation
62F15 Bayesian inference
60G57 Random measures

Software:

BayesDA; PRMLT
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Albert, J. H.; Chib, S., Bayes inference via gibbs sampling of autoregressive time series subject to markov mean and variance shifts, J. Bus. Econom. Statist., 11, 1, 1-15 (1993)
[2] Alon, J.; Sclaroff, S.; Kollios, G.; Pavlovic, V., Discovering clusters in motion time-series data, (IEEE Conf. Computer Vision and Pattern Recognition (2003))
[3] Antoniak, C. E., Mixtures of dirichlet processes with applications to Bayesian nonparametric problems, Ann. Statist., 1152-1174 (1974) · Zbl 0335.60034
[4] Bishop, C. M.; Nasrabadi, N. M., Pattern Recognition and Machine Learning, Vol. 1 (2006), Springer: Springer New York · Zbl 1107.68072
[5] Chib, S., Calculating posterior distributions and modal estimates in markov mixture models, J. Econometrics, 75, 1, 79-97 (1996) · Zbl 0864.62010
[6] Escobar, M. D.; West, M., Bayesian density estimation and inference using mixtures, J. Amer. Statist. Assoc., 90, 430, 577-588 (1995) · Zbl 0826.62021
[7] Fox, E. B.; Sudderth, E. B.; Jordan, M. I.; Willsky, A. S., A sticky hdp-hmm with application to speaker diarization, Ann. Appl. Stat., 5, 2A, 1020-1056 (2011) · Zbl 1232.62077
[8] Gelman, A.; Carlin, J. B.; Stern, H. S.; Rubin, D. B., Bayesian Data Analysis (2003), CRC press
[9] Huo, Q.; Chan, C.; Lee, C. H., Bayesian adaptive learning of the parameters of hidden markov model for speech recognition, IEEE Trans. Speech Audio Process., 3, 5, 334-345 (1995)
[10] Ishwaran, H.; James, L. F., Gibbs sampling methods for stick-breaking priros, J. Amer. Statist. Assoc., 96, 453, 161-173 (2001) · Zbl 1014.62006
[11] Ishwaran, H.; Zarepour, M., Exact and approximate sum representations for the dirichlet process, Canad. J. Statist., 30, 2, 269-283 (2002) · Zbl 1035.60048
[12] Lennox, K. P.; Dahl, D. B.; Vannucci, M.; Day, R.; Tsai, J. W., A dirichlet process mixture of hidden markov models for protein structure prediction, Ann. Appl. Stat., 4, 2, 916 (2010) · Zbl 1194.62117
[13] Lin, Y., Learning features and segments from waveforms: A statistical model of early phonological acquisition (2005), University of California Los Angeles, (Ph.D. thesis)
[14] Qi, Y.; Paisley, J. W.; Carin, L., Music analysis using hidden markov mixture models, IEEE Trans. Signal Process., 55, 11, 5209-5224 (2007) · Zbl 1390.94373
[15] Robert, C. P.; Celeux, G.; Diebolt, J., Bayesian estimation of hidden markov chains: A stochastic implementation, Statist. Probab. Lett., 16, 1, 77-83 (1993) · Zbl 0783.62062
[16] Robert, C. P.; RydÉn, T.; Titterington, D., Convergence controls for mcmc algorithms, with applications to hidden markov chains, J. Stat. Comput. Simul., 64, 4, 327-355 (1999) · Zbl 0968.62049
[17] Robert, C. P.; Titterington, D., Reparameterization strategies for hidden markov models and Bayesian approaches to maximum likelihood estimation, Stat. Comput., 8, 2, 145-158 (1998)
[18] Rydén, T., Em versus markov chain Monte Carlo for estimation of hidden markov models: A computational perspective, Bayesian Anal., 3, 4, 659-688 (2008) · Zbl 1330.65023
[19] Schliep, A.; Schönhuth, A.; Steinhoff, C., Using hidden markov models to analyze gene expression time course data, Bioinformatics, 19, suppl 1, i255-i263 (2003)
[20] Scott, S. L., Bayesian methods for hidden markov models, Journal of the American Statistical Association, 97, 457 (2002) · Zbl 1073.65503
[21] Scott, S. L., Bayesian analysis of a two-state markov modulated Poisson process, J. Comput. Graph. Statist., 8, 3, 662-670 (1999)
[22] Sethuraman, J., A constructive definition of dirichlet priors, Tech. rep (1991), DTIC Document
[24] West, M.; Escobar, M. D., Hierarchical priors and mixture models, with application in regression and density estimation, (Institute of Statistics and Decision Sciences (1993), Duke University)
[25] Ypma, A.; Heskes, T., Automatic categorization of web pages and user clustering with mixtures of hidden markov models, (WEBKDD 2002-Mining Web Data for Discovering Usage Patterns and Profiles (2003), Springer), 35-49
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.