×

Seven things to remember about hidden Markov models: A tutorial on Markovian models for time series. (English) Zbl 1229.62128

Summary: This paper provides a tutorial on key issues in hidden Markov modeling. Hidden Markov models have become very popular models for time series and longitudinal data in recent years due to a combination of (relative) simplicity and flexibility in adapting the model to novel situations. The tutorial covers the conceptual description of the model, estimation of parameters through maximum likelihood, and ends with an application to real data illustrating the possibilities.

MSC:

62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62M09 Non-Markovian processes: estimation
60J99 Markov processes
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Agresti, A., Categorical data analysis, (Wiley series in probability and mathematical statistics (2002), Wiley-Interscience: Wiley-Interscience Hoboken, NJ) · Zbl 0825.62628
[2] Al-Ani, T., Using hidden Markov models for sleep disordered breathing identification, Simulation Modelling Practice and Theory, 12, 117-128 (2004)
[3] Altman, R. M., Mixed hidden Markov models, Journal of the American Statistical Association, 102, 201-210 (2007) · Zbl 1284.62803
[4] Bartolucci, F., Farcomeni, A., & Pennoni, F. 2010. An overview of latent Markov models for longitudinal categorical data. Arxiv preprint http://arxiv.org/abs/1003.2804; Bartolucci, F., Farcomeni, A., & Pennoni, F. 2010. An overview of latent Markov models for longitudinal categorical data. Arxiv preprint http://arxiv.org/abs/1003.2804 · Zbl 1341.62002
[5] Bartolucci, F.; Lupparelli, M.; Montanari, G., Latent Markov model for longitudinal binary data: an application to the performance evaluation of nursing homes, The Annals of Applied Statistics, 3, 611-636 (2009) · Zbl 1166.62330
[6] Bartolucci, F.; Pennoni, F.; Francis, B., A latent Markov model for detecting patterns of criminal activity, Journal of the Royal Statistical Society: Series A Statistics in Society, 170, 115-132 (2007)
[7] Bartolucci, F.; Solis-Trapala, I. L., Multidimensional latent Markov models in a developmental study of inhibitory control and attentional flexibility in early childhood, Psychometrika, 75, 725-743 (2010) · Zbl 1208.62179
[8] Batchelder, W., An all-or-none theory for learning on both the paired-associate and concept levels, Journal of Mathematical Psychology, 7, 97-117 (1970) · Zbl 0186.53801
[9] Baum, L. E.; Petrie, T., Statistical inference for probabilistic functions of finite state Markov chains, Annals of Mathematical Statistics, 67, 1554-1563 (1966) · Zbl 0144.40902
[10] Bijleveld, C. C.; Mooijaart, A., Latent Markov modelling of recidivism data, Statistica Neerlandica, 57, 305-320 (2003) · Zbl 1090.60512
[11] Böckenholt, U., Measuring change: mixed Markov models for ordinal panel data, British Journal of Mathematical and Statistical Psychology, 52, 125-136 (1999)
[12] Böckenholt, U., A latent Markov model for the analysis of longitudinal data collected in continuous time: states, durations, and transitions, Psychological Methods, 10, 65-83 (2005)
[13] Bollen, K. A., Latent variables in psychology and the social sciences, Annual Review of Psychology, 53, 605-634 (2002)
[14] Bower, G. H.; Trabasso, T., Concept identification, (Atkinson, R. C., Studies in mathematical psychology (1964), Stanford University Press), 32-94
[15] Bulla, J.; Berzel, A., Computational issues in parameter estimation for stationary hidden Markov models, Computational Statistics, 23, 1-18 (2008)
[16] Bulla, J.; Bulla, I., Stylized facts of financial time series and hidden semi-Markov models, Computational Statistics & Data Analysis, 51, 2192-2209 (2006) · Zbl 1157.62518
[17] Bulla, J., Mergner, S., Bulla, I., Sesboue, A., & Chesneau, C. 2011. Markov-switching asset allocation: do profitable strategies exist? Journal of Asset Management; Bulla, J., Mergner, S., Bulla, I., Sesboue, A., & Chesneau, C. 2011. Markov-switching asset allocation: do profitable strategies exist? Journal of Asset Management
[18] Cappe, O.; Moulines, E.; Ryden, T., Inference in hidden Markov models, (Springer series in statistics (2005), Springer: Springer New York) · Zbl 1080.62065
[19] Chung, H.; Walls, T.; Park, Y., A latent transition model with logistic regression, Psychometrika, 72, 413-435 (2007) · Zbl 1288.62177
[20] Dempster, A. P.; Laird, N. M.; Rubin, D. B., Maximum likelihood from incomplete data via the em algorithm, Journal of the Royal Statistical Society, 39, 1-38 (1978), Series B Methodological · Zbl 0364.62022
[21] Dutilh, G.; Wagenmakers, E.-J.; Visser, I.; van der Maas, H. L.J., A phase transition model for the speed-accuracy trade-off in response time experiments, Cognitive Science, 35, 211-250 (2011)
[22] Ephraim, Y.; Merhav, N., Hidden Markov processes, IEEE Transactions on Information Theory, 48, 1518-1569 (2002) · Zbl 1061.94560
[23] Flexer, A.; Sykacek, P.; Rezek, I.; Dorffner, G., An automatic, continuous and probabilistic sleep stager based on a hidden Markov model, Applied Artificial Intelligence, 16, 199-207 (2002)
[24] Forney Jr, G. D., The viterbi algorithm, Proceedings of the IEEE, 61, 268-278 (1973)
[25] Frühwirth-Schnatter, S., Finite mixture and Markov switching models, (Springer series in statistics (2006), Springer) · Zbl 1108.62002
[26] Ghahramani, Z., An introduction to hidden Markov models and Bayesian networks, IJPRAI, 15, 9-42 (2001)
[27] Ghysels, E., On the periodic structure of the business cycle, Journal of Business and Economic Statistics, 12, 289-298 (1994)
[28] Giudici, P.; Ryden, T.; Vandekerkhove, P., Likelihood-ratio tests for hidden Markov models, Biometrics, 56, 742-747 (2000) · Zbl 1060.62550
[29] Guedon, Y., Exploring the state sequence space for hidden Markov and semi-Markov chains, Computational Statistics & Data Analysis, 51, 2379-2409 (2007) · Zbl 1161.62412
[30] Hamaker, E., Using information criteria to determine the number of regimes in threshold autoregressive models, Journal of Mathematical Psychology, 53, 518-529 (2009)
[31] Hamilton, J., A new approach to the economic analysis of nonstationary time series and the business cycle, Econometrica: Journal of the Econometric Society, 57, 357-384 (1989) · Zbl 0685.62092
[32] Harte, D. 2010. HiddenMarkov: Hidden Markov Models. \(R\); Harte, D. 2010. HiddenMarkov: Hidden Markov Models. \(R\)
[33] Hughes, J.; Guttorp, P., A class of stochastic models for relating synoptic atmospheric patterns to regional hydrologic phenomena, Water Resources Research, 30, 1535-1546 (1994)
[34] Hughes, J.; Guttorp, P.; Charles, S., A non-homogeneous hidden Markov model for precipitation occurrence, Journal of the Royal Statistical Society: Series C Applied Statistics, 48, 15-30 (1999) · Zbl 0920.62141
[35] Jackson, C. 2010. msm: Multi-state Markov and hidden Markov models in continuous time. \(R\); Jackson, C. 2010. msm: Multi-state Markov and hidden Markov models in continuous time. \(R\)
[36] Jelinek, F., Continuous speech recognition by statistical methods, Proceedings of the IEEE, 64, 532-556 (1976)
[37] Juang, B.; Rabiner, L., The segmental \(K\)-means algorithm for estimating parameters of hidden Markov models, Acoustics, Speech and Signal Processing, IEEE Transactions on, 38, 1639-1641 (1990) · Zbl 0708.62076
[38] Kaplan, D., An overview of Markov chain methods for the study of stage-sequential developmental processes, Developmental Psychology, 44, 457-467 (2008)
[39] Kemeny, J. G.; Snell, J., Finite Markov chains (1960), Princeton: Princeton Van Nostrand · Zbl 0112.09802
[40] Kim, C.-J., Dynamic linear models with Markov-switching, Journal of Econometrics, 60, 1-22 (1994) · Zbl 0795.62104
[41] Kintsch, W.; Morris, C. J., Application of a Markov model to free recall and recognition, Journal of Experimental Psychology, 69, 200-206 (1965)
[42] Krogh, A., An introduction to hidden Markov models for biological sequences, (Salzberg, S. L.; Searls, D. B.; Kasif, S., Computational methods in molecular biology (1998), Elsevier: Elsevier Amsterdam), 45-63, (chapter 4) · Zbl 0951.92010
[43] Langeheine, R., Manifest and latent Markov chain models for categorical panel data, Journal of Educational and Behavioral Statistics, 13, 299 (1988)
[44] Langeheine, R.; Van de Pol, F., A unifying framework for Markov modeling in discrete space and discrete time, Sociological Methods and Research, 18, 416-441 (1990)
[45] Langeheine, R.; Van de Pol, F., Fitting higher order Markov chains, Methods of Psychological Research Online, 5, 32-55 (2000)
[46] Leroux, B. G.; Puterman, M. L., Maximum-penalized-likelihood estimation for independent and Markov-dependent mixture models, Biometrics, 48, 545-548 (1992)
[47] Liechty, J.; Pieters, R.; Wedel, M., Global and local covert visual attention: evidence from a Bayesian hidden Markov model, Psychometrika, 68, 519-541 (2003) · Zbl 1306.62468
[48] Lystig, T. C.; Hughes, J. P., Exact computation of the observed information matrix for hidden Markov models, Journal of Computational and Graphical Statistics (2002)
[49] Mackay, R. J., Estimating the order of a hidden Markov model, Canadian Journal of Statistics, 30, 573-589 (2002) · Zbl 1018.62062
[50] Mackay Altman, R., Assessing the goodness-of-fit of hidden Markov models, Biometrics, 60, 444-450 (2004) · Zbl 1274.62708
[51] McLachlan, G. J.; Krishnan, T., The EM algorithm and extensions (1997), John Wiley & sons: John Wiley & sons New York · Zbl 0882.62012
[52] McLachlan, G. J.; Peel, D., Finite mixture models, (Wiley series in probability and mathematical statistics (2000), Wiley-Interscience: Wiley-Interscience New York etc) · Zbl 0963.62061
[53] Miller, G. A., Finite Markov processes in psychology, Psychometrika, 17, 149-167 (1952) · Zbl 0049.37801
[54] Molenaar, P., A manifesto on psychology as ideographic science: bringing the person back into scientific psychology, this time forever, Measurement, 2, 201-218 (2004)
[55] Otterpohl, J., A constrained HMM-based approach to the estimation of perceptual switching dynamics in pigeons, Neurocomputing, 38-40, 1495-1501 (2001)
[56] Pew, R., The speed-accuracy operating characteristic, Acta Psychologica, 30, 16-26 (1969)
[57] Qin, F., Restoration of single-channel currents using the segmental \(k\)-means method based on hidden Markov modeling, Biophysical Journal, 86, 1488-1501 (2004)
[58] Rabiner, L. R., A tutorial on hidden Markov models and selected applications in speech recognition, Proceedings of IEEE, 77, 267-295 (1989)
[59] \(RRR\); \(RRR\)
[60] Schmittmann, V. D.; Dolan, C. V.; van der Maas, H. L.J.; Neale, M. C., Discrete latent Markov models for normally distributed response data, Multivariate Behavioral Research, 40, 461-488 (2005)
[61] Schmittmann, V. D.; Visser, I.; Raijmakers, M. E.J., Multiple learning modes in the development of rule-based category-learning task performance, Neuropsychologia, 44, 2079-2091 (2006)
[62] Schwarz, G., Estimating the dimension of a model, Annals of Statistics, 6, 461-464 (1978) · Zbl 0379.62005
[63] Taramasco, O. 2009. RHmm: hidden Markov models simulations and estimations. \(R\); Taramasco, O. 2009. RHmm: hidden Markov models simulations and estimations. \(R\)
[64] Titman, A. C.; Sharples, L. D., A general goodness-of-fit test for Markov and hidden Markov models, Statistics in Medicine, 27, 2177-2195 (2008)
[65] Turner, R., Direct maximization of the likelihood of a hidden Markov model, Computational Statistics & Data Analysis, 52, 4147-4160 (2008) · Zbl 1452.62606
[66] Vermunt, J. K.; Langeheine, R.; Böckenholt, U., Discrete-time discrete-state latent Markov modles with time-constant and time-varying covariates, Journal of Educational and Behavioral Statistics, 24, 179-207 (1999)
[67] Visser, I., Book review of Zucchini & MacDonald: hidden Markov models for time series: an introduction using \(R\), Journal of Mathematical Psychology, 54, 509-511 (2010)
[68] Visser, I.; Raijmakers, M. E.J.; Molenaar, P. C.M., Confidence intervals for hidden Markov model parameters, British Journal of Mathematical and Statistical Psychology, 53, 317-327 (2000)
[69] Visser, I.; Raijmakers, M. E.J.; Molenaar, P. C.M., Fitting hidden Markov models to psychological data, Scientific Programming, 10, 185-199 (2002)
[70] Visser, I.; Raijmakers, M. E.J.; Molenaar, P. C.M., Characterizing sequence knowledge using online measures and hidden Markov models, Memory & Cognition, 35, 1502-1517 (2007)
[71] Visser, I.; Speekenbrink, M., depmixs4: An \(R\)-package for hidden Markov models, Journal of Statistical Software, 36, 1-21 (2010), \(R\) package, current version available from CRAN
[72] Viterbi, A., Error bounds for convolutional codes and an asymptotically optimum decoding algorithm, IEEE Transactions on Information Theory, 13, 260-269 (1967) · Zbl 0148.40501
[73] Wickelgren, W., Speed-accuracy tradeoff and information processing dynamics, Acta Psychologica, 41, 67-85 (1977)
[74] Wickens, T. D., Models for behavior: stochastic processes in psychology (1982), W. H. Freeman and Company: W. H. Freeman and Company San Francisco · Zbl 0538.92024
[75] Wiggins, L. M., Panel analysis (1973), Elsevier Scientific Publishing Company
[76] Zucchini, W., An introduction to model selection, Journal of Mathematical Psychology, 44, 41-61 (2000) · Zbl 0949.62092
[77] Zucchini, W.; MacDonald, I., Hidden Markov models for time series: An introduction using R. Number 110 in monographs on statistics and applied probability (2009), CRC Press: CRC Press Boca Raton
[78] Zucchini, W.; Raubenheimer, D.; MacDonald, I. L., Modeling time series of animal behavior by means of a latent-state model with feedback, Biometrics, 64, 807-815 (2008) · Zbl 1170.62408
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.