×

Semi-parametric dynamic time series modelling with applications to detecting neural dynamics. (English) Zbl 1184.62155

Summary: This paper illustrates novel methods for nonstationary time series modeling along with their applications to selected problems in neuroscience. These methods are semi-parametric in that inferences are derived by combining sequential Bayesian updating with a nonparametric change-point test. As a test statistic, we propose a Kullback-Leibler (KL) divergence between posterior distributions arising from different sets of data. A closed form expression of this statistic is derived for exponential family models, whereas standard Markov chain Monte Carlo output is used to approximate its value and its critical region for more general models. The behavior of one-step ahead predictive distributions under our semi-parametric framework is described analytically for a dynamic linear time series model. Conditions under which our approach reduces to fully parametric state-space modeling are also illustrated.
We apply our methods to estimating the functional dynamics of a wide range of neural data, including multi-channel electroencephalogram recordings, longitudinal behavioral experiments and in-vivo multiple spike trains recordings. The estimated dynamics are related to the presentation of visual stimuli, to the evaluation of a learning performance and to changes in the functional connections between neurons over a sequence of experiments.

MSC:

62M10 Time series, auto-correlation, regression, etc. in statistics (GARCH)
62P10 Applications of statistics to biology and medical sciences; meta analysis
62F15 Bayesian inference
92C20 Neural biology
62L10 Sequential statistical analysis
62G10 Nonparametric hypothesis testing
65C40 Numerical analysis or methods applied to Markov chains
62G08 Nonparametric regression and quantile regression
PDF BibTeX XML Cite
Full Text: DOI arXiv

References:

[1] Akaike, H. (1978). On the likelihood of a time series model. The Statistician 27 217-235.
[2] Akaike, H. (1981). Likelihood of a model and information criteria. J. Econometrics 16 3-14. · Zbl 0457.62032
[3] Albert, J. H. and Chib, S. (1993). Bayes inference via Gibbs sampling for autoregressive time series subject to Markov mean and variance shifts. J. Bus. Econom. Statist. 11 1-15.
[4] Barbieri, M. M. and Berger, J. O. (2004). Optimal predictive model selection. Ann. Statist. 32 870-897. · Zbl 1092.62033
[5] Barnard, G. A. (1959). Control charts and stochastic processes. J. Roy. Statist. Soc. Ser. B 21 239-271. · Zbl 0089.35102
[6] Bayarri, M. J. and Berger, J. O. (2004). The interplay between Bayesian and frequentist analysis. Statist. Sci. 19 58-80. · Zbl 1062.62001
[7] Bayarri, M. J. and Morales, J. (2003). Bayesian measures of surprise for outlier detection. J. Statist. Plann. Inference 111 3-22. · Zbl 1027.62014
[8] Bélisle, P., Joseph, L., MacGibbon, B., Wolfson, D. B. and du Berger, R. (1998). Change-point analysis of neuron spike train data. Biometrics 54 113-123. · Zbl 1058.62576
[9] Bengtsson, T. and Cavanaugh, J. E. (2006). An improved Akaike information criterion for state-space model selection. Comput. Statist. Data Anal. 50 2635-2654. · Zbl 1446.62235
[10] Berger, J. O. and Bayarri, M. L. (1997). Measures of surprise in Bayesian analysis. ISDS Discussion Paper 46, Duke Univ.
[11] Berger, J. O., Brown, L. and Wolpert, R. L. (1994). A unified conditional frequentist and Bayesian test for fixed and sequential hypothesis testing. Ann. Statist. 22 1787-1807. · Zbl 0824.62002
[12] Bernardo, J. (1979). Expected information as expected utility. Ann. Statist. 7 686-690. · Zbl 0407.62002
[13] Bernardo, J. M. and Smith, A. F. M. (2007). Bayesian Theory . Wiley, Chichester, UK.
[14] Box, G. E. P. (1980). Sampling and Bayes’ inference in scientific modelling and robustness. J. Amer. Statist. Assoc. 143 383-430. · Zbl 0471.62036
[15] Brillinger, D. R. (1988). Some statistical methods for random processes data from seismology and neurophysiology. Ann. Statist. 16 1-54. · Zbl 0637.62089
[16] Brown, E. N. and Barbieri, R. (2006). Dynamic analyses of neural representations using the state-space modeling paradigm. In: The Cell Biology of Addiction (B. Madras, M. Von Zastrow, C. Colvis, J. Rutter, D. Shurtleff and J. Pollock, eds.). Cold Spring Harbor Laboratory Press, New York.
[17] Brown, E., Kass, R. E. and Mitra, P. P. (2004). Multiple neural spike train data analysis: State-of-the-art and future challenges. Nature Neuroscience 7 456-461.
[18] Buzsáki, G. (2004). Large scale recording of neuronal ensembles. Nature Neuroscience 7 446-451.
[19] Cappe, O., Moulines, E. and Ryden, T. (2005). Inference in Hidden Markov Models . Springer, New York. · Zbl 1080.62065
[20] Carlin, B. P., Gelfand, A. E. and Smith, A. F. M. (1992). Hierarchical Bayesian analysis of changepoint problems. App. Statist. 41 389-405. · Zbl 0825.62408
[21] Carota, C., Parmigiani, G. and Polson, N. (1996). Diagnostic measures for model criticism. J. Amer. Statist. Assoc. 91 753-762. · Zbl 0868.62012
[22] Chib, S. (1995). Marginal likelihood from the Gibbs output. J. Amer. Statist. Assoc. 90 1313-1321. · Zbl 0868.62027
[23] Chib, S. (1998). Estimation and comparison of multiple change-point models. J. Econometrics 86 221-241. · Zbl 1045.62510
[24] Critchley, F., Marriott, P. and Salmon, M. (1994). Preferred point geometry and the local differential geometry of the Kullback-Leibler divergence. Ann. Statist. 22 1587-1602. · Zbl 0821.62004
[25] Dawid, A. P. (1984). Present position and potential developments: Some personal views. Statistical theory: The prequential approach. J. Roy. Statist. Soc. Ser. A 147 278-292. · Zbl 0557.62080
[26] Delorme, A., Makeig, S., Fabre-Thorpe, M. and Sejnowski, T. J. (2002). From single trial EEG to brain area dynamics. Neurocomputing 44 1057-1064. · Zbl 1007.68790
[27] Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. Ann. Statist. 7 269-281. · Zbl 0405.62011
[28] Doucet, A., De Freitas, N. and Gordon, N. J. (2001). Sequential Monte Carlo Methods in Practice . Springer-Verlag, New York. · Zbl 0967.00022
[29] Eden, U. T. and Brown, E. N. (2008). Continuous-time filters for state estimation from point process models of neural data. Statist. Sinica . · Zbl 1293.62212
[30] Eden, U. T., Frank, L. M., Barbieri, R., Solo, V. and Brown, E. N. (2004). Dynamic analysis of neural encoding by point process adaptive filtering. Neural Comput. 16 971-998. · Zbl 1054.92008
[31] Fearnhead, P. and Liu, Z. (2007). On-line inference for multiple change points. J. Roy. Statist. Soc. Ser. B 69 589-605.
[32] Ferger, D. (1995). Nonparametric tests for nonstandard change-point problems. Ann. Statist. 23 1848-1861. · Zbl 0843.62054
[33] Fienberg, S. E. (1974). Stochastic models for single neuron firing trains: A survey. Biometrics 30 399-427. · Zbl 0286.92003
[34] Frühwirth-Shnatter, S. (1995). Bayesian model discrimination and Bayes factors for linear Gaussian state-space models. J. Roy. Statist. Soc. Ser. B 1 237-246. · Zbl 0809.62023
[35] Frühwirth-Shnatter, S. (2001). Markov chain Monte Carlo estimation of classical and dynamic switching and mixture models. J. Amer. Statist. Assoc. 96 194-209. · Zbl 1015.62022
[36] Frühwirth-Shnatter, S. (2006). Finite Mixture and Markov Switching Models . Springer, New York. · Zbl 1108.62002
[37] Gamerman, D. (1992). A dynamic approach to the statistical analysis of point processes. Biometrika 79 39-50. · Zbl 0754.62068
[38] Geisser, S. and Eddy, W. F. (1979). A predictive approach to model selection. J. Amer. Statist. Assoc. 74 153-160. · Zbl 0401.62036
[39] Gelfand, A. E. and Dey, E. K. (1994). Bayesian model choice: Asymptotics and exact calculations. J. Amer. Statist. Assoc. 56 501-514. · Zbl 0800.62170
[40] Gelfand, A. E. and Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. J. Amer. Statist. Assoc. 85 398-409. · Zbl 0702.62020
[41] Gelman, A., Meng, X. L. and Stern, H. S. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statist. Sinica 6 733-807. · Zbl 0859.62028
[42] Ghahramani, Z. and Hinton, G. E. (2000). Variational learning for switching state-space models. Neural Comput. 12 831-864.
[43] Goutis, C. and Robert, C. (1998). Model choice in generalised linear models: A Bayesian approach via Kullback-Leibler projections. Biometrika 85 29-37. · Zbl 0903.62061
[44] Gutiérrez-Peña, E. (1997). Moments for the canonical parameter of an exponential family under a conjugate distribution. Biometrika 84 727-732. · Zbl 1090.62513
[45] Guttman, I. (1967). The use of the concept of a future observation in goodness-of-fit problems. J. Roy. Statist. Soc. Ser. B 29 83-100. · Zbl 0158.37305
[46] Hall, P. (1987). On Kullback-Leibler loss and density estimation. Ann. Statist. 15 1491-1519. · Zbl 0678.62045
[47] Hamilton, J. D. (1990). Analysis of time series subject to changes in regime. J. Econometrics 45 39-70. · Zbl 0723.62050
[48] Hamilton, J. D. (1994). Time Series Analysis . Princeton Univ. Press, New Jersey. · Zbl 0831.62061
[49] Han, C. and Carlin, B. P. (2001). Markov chain Monte Carlo methods for computing Bayes factors: A comparative review. J. Amer. Statist. Assoc. 96 1122-1132.
[50] Härdle, W., Lütkepohl, H. and Chen, R. (1997). A review of nonparametric time series analysis. International Statistical Review 65 49-72. · Zbl 0887.62043
[51] Harrison, P. J. and Stevens, C. F. (1976). Bayesian forecasting. J. Roy. Statist. Soc. Ser. B 38 205-247. · Zbl 0349.62062
[52] Hastie, T. (1987). A closer look at the deviance. Amer. Statist. 41 16-20. · Zbl 0607.62084
[53] Iyengar, S. (2001). The analysis of multiple neural spike trains. In Advances in Methodological and Applied Aspects of Probability and Statistics (N. Bolakrishnan, ed.) 507-524. Taylor and Francis, New York. · Zbl 1271.62259
[54] Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Journal of Basic Engineering 82 35-45.
[55] Kass, R. E., Ventura, V. and Brown, E. (2005). Statistical issues in the analysis of neuronal data. Journal of Neurophysiology 94 8-25.
[56] Kemp, K. W. (1957). Formulae for calculating the operating characteristic and the average sample number of some sequential tests. J. Roy. Statist. Soc. Ser. B 20 379-386. · Zbl 0085.35306
[57] Kendrick, K. M., da Costa, A. P., Leigh, A. E., Hinton, M. R. and Peirce, J. W. (2001). Sheep don’t forget a face. Nature 414 165-166.
[58] Kim, C. J. (1994). Dynamic linear models with Markov-switching. J. Econometrics 60 1-22. · Zbl 0795.62104
[59] Koyama, S., Perez-Bolde, L. C. and Kass, R. E. (2008). Approximate methods for state-space models: The Laplace-Gaussian filter. Submitted for publication.
[60] Kuhnert, P. M., Mergesen, K. and Tesar, P. (2003). Bridging the gap between different statistical approaches: An integrated framework for modelling. International Statistical Review 71 335-368. · Zbl 1114.62307
[61] Kullback, S. (1997). Information Theory and Statistics . Dover, New York. · Zbl 0897.62003
[62] Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. Ann. Math. Statist. 22 79-86. · Zbl 0042.38403
[63] Lindley, D. (1956). On a measure of the information provided by an experiment. Ann. Math. Statist. 27 986-1005. · Zbl 0073.14103
[64] Loader, C. R. (1996). Change point estimation using nonparametric regression. Ann. Statist. 24 1667-1678. · Zbl 0867.62033
[65] Makeig, S., Westerfield, M., Jung, T. P., Enghoff, S., Towsend, J., Courchesne, E. and Sejnowski, T. J. (2002). Dynamic brain sources of visual evoked responses. Science 295 690-694.
[66] Marjoram, P., Molitor, J., Plagnol, V. and Tavaré, S. (2003). Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. USA 100 15324-15328.
[67] McCulloch, R. E. (1988). Information and the likelihood function in exponential families. Amer. Statist. 42 73-75.
[68] McCulloch, R. E. and Tsay, R. S. (1994). Statistical analysis of economic time series via Markov switching models. J. Time Ser. Anal. 15 523-539. · Zbl 0800.62549
[69] Meng, X. L. (1994). Posterior predictive p -values. Ann. Statist. 22 1142-1160. · Zbl 0820.62027
[70] Mira, A. and Petrone, S. (1996). Bayesian hierarchical nonparametric inference for change point problems. Bayesian Statistics 5 693-703.
[71] Muller, H. G. (1992). Change-points in nonparametric regression analysis. Ann. Statist. 20 737-761. · Zbl 0783.62032
[72] Newton, M. A. and Raftery, A. E. (1994). Approximate Bayesian inference by the weighted likelihood bootstrap. J. Roy. Statist. Soc. Ser. B 56 3-56. · Zbl 0788.62026
[73] O’Hagan, T. and Forster, J. (1999). Kendall’s Advanced Theory of Statistics 2B . Arnold, London, UK. · Zbl 1058.62002
[74] Okatan, M., Wilson, M. A. and Brown, E. N. (2005). Analysing functional connectivity using a network likelihood model of ensemble neural spiking activity. Neural Comput. 17 1927-1961. · Zbl 1076.92011
[75] Page, E. S. (1954). An improvement to Wald’s approximation for some properties of sequential tests. J. Roy. Statist. Soc. Ser. B 16 136-139. · Zbl 0058.35402
[76] Page, E. S. (1955). A test for a change in a parameter occurring at an unknown point. Biometrika 42 523-527. · Zbl 0067.11602
[77] Page, E. S. (1961). Cumulative sum charts. Technometrics 3 1-9.
[78] Plesser, H. E. and Gerstner, W. (2000). Niose in integrate-and-fire neurons: From stochastic input to escape rates. Neural Comput. 12 367-384.
[79] Rao, R. P. N. (2005). Hierarchical Bayesian inference in networks of spiking neurons. In Advances in NIPS 17 . MIT Press, MA.
[80] Rigat, F., de Gunst, M. and ven Pelt, J. (2006). Bayesian modelling and analysis of spatio-temporal neuronal networks. Bayesian Anal. 1 733-764. · Zbl 1331.62161
[81] Robert, C. P., Celeux, G. and Diebolt, J. (1993). Bayesian estimation of hidden Markov chains: A stochastic implementation. Statist. Probab. Lett. 16 77-83. · Zbl 0783.62062
[82] Robinson, P. M. (1983). Non-parametric estimation for time series models. J. Time Ser. Anal. 4 185-208. · Zbl 0544.62082
[83] Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Statist. 12 1151-1172. · Zbl 0555.62010
[84] San Martini, A. and Spezzaferri, F. (1984). A predictive model selection criterion. J. Roy. Statist. Soc. Ser. B 46 296-383. · Zbl 0566.62004
[85] Shumway, R. H. and Stoffer, D. S. (1991). Dynamic linear models with switching. J. Amer. Statist. Assoc. 86 763-769.
[86] Smith, A. C., Loren, M. F., Wyrth, S., Yanike, M., Hu, D., Kubota, Y., Graybiel, A. M., Suzuki, W. A. and Brown, E. M. (2004). Dynamic analysis of learning in behavioural experiments. Journal of Neuroscience 24 447-461.
[87] Smith, A. F. M. (1975). A Bayesian approach to inference about a change-point in a sequence of random variables. Biometrika 62 407-416. · Zbl 0321.62041
[88] Smith, A. F. M. and Roberts, G. O. (1993). Bayesian computations via the Gibbs sampler and related Markov chain Monte Carlo methods. J. Roy. Statist. Soc. Ser. B 55 3-23. · Zbl 0779.62030
[89] Smith, J. Q. (1990). Non-linear state space models with partially specified distributions on states. J. Forecast. 9 137-149.
[90] Smith, J. Q. (1992). A comparison of the characteristics of some Bayesian forecasting models. International Statistical Reviews 60 75-85. · Zbl 0755.62033
[91] Spiegelhalter, D. J., Best, N. G., Carlin, P. B. and van der Linde, A. (2002). Bayesian measures of model complexity and fit. J. Roy. Statist. Soc. Ser. B 64 583-639. · Zbl 1067.62010
[92] Srinivansan, L., Eden, U. T., Willsky, A. S. and Brown, E. N. (2006). A state-space analysis for reconstruction of goal-directed movements using neural signals. Neural Comput. 18 2465-2494. · Zbl 1106.92017
[93] Stephens, D. A. (1994). Bayesian retrospective multiple-changepoint identification. App. Statist. 43 159-178. · Zbl 0825.62412
[94] Stone, M. (1959). Application of a measure of information to the design and comparison of regression experiments. Ann. Math. Statist. 30 55-70. · Zbl 0094.13602
[95] Tierney, L. (1994). Markov chains for exploring posterior distributions. Ann. Statist. 22 1701-1762. · Zbl 0829.62080
[96] Tierney, L. and Kadane, J. B. (1986). Accurate approximations for posterior moments and marginal densities. J. Amer. Statist. Assoc. 81 84-86. · Zbl 0587.62067
[97] Truccolo, W., Eden, U. T., Fellows, M. R., Donoghue, J. P. and Brown, E. N. (2005). A point process framework for relating neural spiking activity to spiking history, neural ensemble and extrinsic covariate effects. Journal of Neurophysiology 93 1074-1089.
[98] West, M. (1986). Bayesian model monitoring. J. Roy. Statist. Soc. Ser. B 48 70-78. · Zbl 0593.62078
[99] West, M. and Harrison, P. J. (1986). Monitoring and adaptation in Bayesian forecasting models. J. Amer. Statist. Assoc. 81 741-750. · Zbl 0648.62098
[100] West, M. and Harrison, P. J. (1997). Bayesian Forecasting and Dynamic Models , 2nd ed. Springer, New York. · Zbl 0871.62026
[101] West, M., Harrison, P. J. and Migon, H. S. (1985). Dynamic generalised linear models and Bayesian forecasting. J. Amer. Statist. Assoc. 80 73-83. · Zbl 0568.62032
[102] Wirth, S., Yanike, M., Loren, M. F., Smith, A. C., Brown, E. M. and Suzuki, W. A. (2003). Single neurons in the monkey hippocampus and learning of new associations. Science 300 1578-1584.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.