×

On particle methods for parameter estimation in state-space models. (English) Zbl 1332.62096

Summary: Nonlinear non-Gaussian state-space models are ubiquitous in statistics, econometrics, information engineering and signal processing. Particle methods, also known as Sequential Monte Carlo (SMC) methods, provide reliable numerical approximations to the associated state inference problems. However, in most applications, the state-space model of interest also depends on unknown static parameters that need to be estimated from the data. In this context, standard particle methods fail and it is necessary to rely on more sophisticated algorithms. The aim of this paper is to present a comprehensive review of particle methods that have been proposed to perform static parameter estimation in state-space models. We discuss the advantages and limitations of these methods and illustrate their performance on simple models.

MSC:

62F15 Bayesian inference
62M05 Markov processes: estimation; hidden Markov models
62M20 Inference from stochastic processes and prediction
PDFBibTeX XMLCite
Full Text: DOI arXiv Euclid

References:

[1] Alspach, D. and Sorenson, H. (1972). Nonlinear Bayesian estimation using Gaussian sum approximations. IEEE Trans. Automat. Control 17 439-448. · Zbl 0264.93023 · doi:10.1109/TAC.1972.1100034
[2] Andrieu, C., De Freitas, J. F. G. and Doucet, A. (1999). Sequential MCMC for Bayesian model selection. In Proc. IEEE Workshop Higher Order Statistics 130-134. IEEE, New York.
[3] Andrieu, C., Doucet, A. and Holenstein, R. (2010). Particle Markov chain Monte Carlo methods. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 269-342. · Zbl 1184.65001 · doi:10.1111/j.1467-9868.2009.00736.x
[4] Andrieu, C., Doucet, A. and Tadić, V. B. (2005). On-line parameter estimation in general state-space models. In Proc. 44 th IEEE Conf. on Decision and Control 332-337. IEEE, New York.
[5] Benveniste, A., Métivier, M. and Priouret, P. (1990). Adaptive Algorithms and Stochastic Approximations. Applications of Mathematics ( New York ) 22 . Springer, Berlin. · Zbl 0752.93073
[6] Briers, M., Doucet, A. and Maskell, S. (2010). Smoothing algorithms for state-space models. Ann. Inst. Statist. Math. 62 61-89. · Zbl 1422.62297 · doi:10.1007/s10463-009-0236-2
[7] Briers, M., Doucet, A. and Singh, S. S. (2005). Sequential auxiliary particle belief propagation. In Proc. Conf. Fusion . Philadelphia, PA.
[8] Cappé, O. (2009). Online sequential Monte Carlo EM algorithm. In Proc. 15 th IEEE Workshop on Statistical Signal Processing 37-40. IEEE, New York.
[9] Cappé, O. (2011). Online EM algorithm for hidden Markov models. J. Comput. Graph. Statist. 20 728-749. · doi:10.1198/jcgs.2011.09109
[10] Cappé, O. and Moulines, E. (2009). On-line expectation-maximization algorithm for latent data models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 71 593-613. · Zbl 1250.62015 · doi:10.1111/j.1467-9868.2009.00698.x
[11] Cappé, O., Moulines, E. and Rydén, T. (2005). Inference in Hidden Markov Models . Springer, New York. · Zbl 1080.62065
[12] Carpenter, J., Clifford, P. and Fearnhead, P. (1999). An improved particle filter for non-linear problems. IEE Proceedings-Radar , Sonar and Navigation 146 2-7.
[13] Carvalho, C. M., Johannes, M. S., Lopes, H. F. and Polson, N. G. (2010). Particle learning and smoothing. Statist. Sci. 25 88-106. · Zbl 1328.62541 · doi:10.1214/10-STS325
[14] Cérou, F., Del Moral, P. and Guyader, A. (2011). A nonasymptotic theorem for unnormalized Feynman-Kac particle models. Ann. Inst. Henri Poincaré , B Probab. Stat. 47 629-649. · Zbl 1233.60047 · doi:10.1214/10-AIHP358
[15] Chen, R. and Liu, J. S. (2000). Mixture Kalman filters. J. R. Stat. Soc. Ser. B. Stat. Methodol. 62 493-508. · Zbl 0953.62100 · doi:10.1111/1467-9868.00246
[16] Chopin, N. (2002). A sequential particle filter method for static models. Biometrika 89 539-551. · Zbl 1036.62062 · doi:10.1093/biomet/89.3.539
[17] Chopin, N. (2004). Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference. Ann. Statist. 32 2385-2411. · Zbl 1079.65006 · doi:10.1214/009053604000000698
[18] Chopin, N., Iacobucci, A., Marin, J. M., Mengersen, K., Robert, C. P., Ryder, R. and Schäufer, C. (2011). On particle learning. In Bayesian Statistics 9 (J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D., Heckerman A. F. M. Smith and M., West, eds.) 317-360. Oxford Univ. Press, Oxford.
[19] Chopin, N., Jacob, P. E. and Papaspiliopoulos, O. (2013). \(\mathrm{SMC}^{2}\): An efficient algorithm for sequential analysis of state space models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 75 397-426. · doi:10.1111/j.1467-9868.2012.01046.x
[20] Coquelin, P. A., Deguest, R. and Munos, R. (2009). Sensitivity analysis in HMMs with application to likelihood maximization. In Proc. 22 th Conf. NIPS . Vancouver.
[21] Dahlin, J., Lindsten, F. and Schön, T. B. (2015). Particle Metropolis-Hastings using gradient and Hessian information. Stat. Comput. 25 81-92. · Zbl 1331.62134 · doi:10.1007/s11222-014-9510-0
[22] DeJong, D. N., Liesenfeld, R., Moura, G. V., Richard, J.-F. and Dharmarajan, H. (2013). Efficient likelihood evaluation of state-space representations. Rev. Econ. Stud. 80 538-567. · Zbl 1263.91036 · doi:10.1093/restud/rds040
[23] Del Moral, P. (2004). Feynman-Kac Formulae : Genealogical and Interacting Particle Systems with Applications . Springer, New York. · Zbl 1130.60003
[24] Del Moral, P., Doucet, A. and Singh, S. S. (2009). Forward smoothing using sequential Monte Carlo. Technical Report 638, CUED-F-INFENG, Cambridge Univ. Preprint. Available at . arXiv:1012.5390
[25] Del Moral, P., Doucet, A. and Singh, S. S. (2010). A backward particle interpretation of Feynman-Kac formulae. ESAIM Math. Model. Numer. Anal. 44 947-975. · Zbl 1209.65009 · doi:10.1051/m2an/2010048
[26] Del Moral, P., Doucet, A. and Singh, S. S. (2015). Uniform stability of a particle approximation of the optimal filter derivative. SIAM J. Control Optim. 53 1278-1304. · Zbl 1349.65010 · doi:10.1137/140993703
[27] Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. Ser. B 39 1-38. · Zbl 0364.62022
[28] Douc, R., Garivier, A., Moulines, E. and Olsson, J. (2011). Sequential Monte Carlo smoothing for general state space hidden Markov models. Ann. Appl. Probab. 21 2109-2145. · Zbl 1237.60026 · doi:10.1214/10-AAP735
[29] Douc, R., Moulines, E. and Ritov, Y. (2009). Forgetting of the initial condition for the filter in general state-space hidden Markov chain: A coupling approach. Electron. J. Probab. 14 27-49. · Zbl 1189.93137 · doi:10.1214/EJP.v14-593
[30] Doucet, A., De Freitas, J. F. G. and Gordon, N. J., eds. (2001). Sequential Monte Carlo Methods in Practice . Springer, New York. · Zbl 0967.00022
[31] Doucet, A., Godsill, S. J. and Andrieu, C. (2000). On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10 197-208.
[32] Doucet, A. and Johansen, A. M. (2011). A tutorial on particle filtering and smoothing: Fifteen years later. In The Oxford Handbook of Nonlinear Filtering 656-704. Oxford Univ. Press, Oxford. · Zbl 1513.60043
[33] Doucet, A., Pitt, M. K., Deligiannidis, G. and Kohn, R. (2015). Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator. Biometrika 102 295-313. · Zbl 1452.62055 · doi:10.1093/biomet/asu075
[34] Elliott, R. J., Aggoun, L. and Moore, J. B. (1995). Hidden Markov Models : Estimation and Control. Applications of Mathematics ( New York ) 29 . Springer, New York. · Zbl 0819.60045
[35] Elliott, R. J., Ford, J. J. and Moore, J. B. (2000). On-line consistent estimation of hidden Markov models. Technical report, Dept. Systems Engineering, Australian National Univ., Canberra.
[36] Fearnhead, P. (2002). Markov chain Monte Carlo, sufficient statistics, and particle filters. J. Comput. Graph. Statist. 11 848-862. · doi:10.1198/106186002835
[37] Fearnhead, P. and Meligkotsidou, L. (2014). Augmentation schemes for particle MCMC. Preprint. Available at . arXiv:1408.6980 · Zbl 1356.65025 · doi:10.1007/s11222-015-9603-4
[38] Fearnhead, P., Wyncoll, D. and Tawn, J. (2010). A sequential smoothing algorithm with linear computational cost. Biometrika 97 447-464. · Zbl 1406.62093 · doi:10.1093/biomet/asq013
[39] Fernández-Villaverde, J. and Rubio-Ramírez, J. F. (2007). Estimating macroeconomic models: A likelihood approach. Rev. Econ. Stud. 74 1059-1087. · Zbl 1206.91064 · doi:10.1111/j.1467-937X.2007.00437.x
[40] Flury, T. and Shephard, N. (2009). Learning and filtering via simulation: Smoothly jittered particle filters. Economics Series Working Papers 469.
[41] Flury, T. and Shephard, N. (2011). Bayesian inference based only on simulated likelihood: Particle filter analysis of dynamic economic models. Econometric Theory 27 933-956. · Zbl 1226.62021 · doi:10.1017/S0266466610000599
[42] Ford, J. J. (1998). Adaptive hidden Markov model estimation and applications. Ph.D. thesis, Dept. Systems Engineering, Australian National Univ., Canberra. Available at .
[43] Fulop, A. and Li, J. (2013). Efficient learning via simulation: A marginalized resample-move approach. J. Econometrics 176 146-161. · Zbl 1284.62516 · doi:10.1016/j.jeconom.2013.05.002
[44] Gilks, W. R. and Berzuini, C. (2001). Following a moving target-Monte Carlo inference for dynamic Bayesian models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 63 127-146. · Zbl 0976.62021 · doi:10.1111/1467-9868.00280
[45] Godsill, S. J., Doucet, A. and West, M. (2004). Monte Carlo smoothing for nonlinear times series. J. Amer. Statist. Assoc. 99 156-168. · Zbl 1089.62517 · doi:10.1198/016214504000000151
[46] Gordon, N. J., Salmond, D. J. and Smith, A. F. M. (1993). Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F , Comm. , Radar , Signal. Proc. 140 107-113.
[47] Higuchi, T. (2001). Self-organizing time series model. In Sequential Monte Carlo Methods in Practice. Stat. Eng. Inf. Sci. 429-444. Springer, New York. · Zbl 1056.93579 · doi:10.1007/978-1-4757-3437-9_20
[48] Hürzeler, M. and Künsch, H. R. (1998). Monte Carlo approximations for general state-space models. J. Comput. Graph. Statist. 7 175-193.
[49] Hürzeler, M. and Künsch, H. R. (2001). Approximating and maximising the likelihood for a general state-space model. In Sequential Monte Carlo Methods in Practice. Stat. Eng. Inf. Sci. 159-175. Springer, New York. · Zbl 1056.93580 · doi:10.1007/978-1-4757-3437-9_8
[50] Ionides, E. L., Bhadra, A., Atchadé, Y. and King, A. (2011). Iterated filtering. Ann. Statist. 39 1776-1802. · Zbl 1220.62103 · doi:10.1214/11-AOS886
[51] Ionides, E. L., Bretó, C. and King, A. A. (2006). Inference for nonlinear dynamical systems. Proc. Natl. Acad. Sci. USA 103 18438-18443.
[52] Kim, S., Shephard, N. and Chib, S. (1998). Stochastic volatility: Likelihood inference and comparison with ARCH models. Rev. Econ. Stud. 65 361-393. · Zbl 0910.90067 · doi:10.1111/1467-937X.00050
[53] Kitagawa, G. (1996). Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Comput. Graph. Statist. 5 1-25.
[54] Kitagawa, G. (1998). A self-organizing state-space model. J. Amer. Statist. Assoc. 93 1203-1215.
[55] Kitagawa, G. (2014). Computational aspects of sequential Monte Carlo filter and smoother. Ann. Inst. Statist. Math. 66 443-471. · Zbl 1334.65025 · doi:10.1007/s10463-014-0446-0
[56] Kitagawa, G. and Sato, S. (2001). Monte Carlo smoothing and self-organising state-space model. In Sequential Monte Carlo Methods in Practice. Stat. Eng. Inf. Sci. 177-195. Springer, New York. · Zbl 1056.93581 · doi:10.1007/978-1-4757-3437-9_9
[57] Klaas, M., Briers, M., De Freitas, N., Doucet, A., Maskell, S. and Lang, D. (2006). Fast particle smoothing: If I had a million particles. In Proc. International Conf. Machine Learning 481-488. Pittsburgh, PA.
[58] Künsch, H. R. (2013). Particle filters. Bernoulli 19 1391-1403. · Zbl 1275.93058 · doi:10.3150/12-BEJSP07
[59] Lee, A. (2008). Towards smoother multivariate particle filters. M.Sc. Computer Science, Univ. British Columbia, Vancouver, BC.
[60] Lee, A. and Whiteley, N. (2014). Forest resampling for distributed sequential Monte Carlo. Preprint. Available at . arXiv:1406.6010
[61] Lee, D. S. and Chia, K. K. (2002). A particle algorithm for sequential Bayesian parameter estimation and model selection. IEEE Trans. Signal Process. 50 326-336.
[62] Le Corff, S. and Fort, G. (2013). Online expectation maximization based algorithms for inference in hidden Markov models. Electron. J. Stat. 7 763-792. · Zbl 1336.62090 · doi:10.1214/13-EJS789
[63] Le Corff, S. and Fort, G. (2013). Convergence of a particle-based approximation of the block online expectation maximization algorithm. ACM Trans. Model. Comput. Simul. 23 Art. 2, 22. · Zbl 1336.62090 · doi:10.1145/2414416.2414418
[64] Le Gland, F. and Mevel, M. (1997). Recursive estimation in hidden Markov models. In Proc. 36 th IEEE Conf. Decision and Control 3468-3473. San Diego, CA.
[65] Lin, M., Chen, R. and Liu, J. S. (2013). Lookahead strategies for sequential Monte Carlo. Statist. Sci. 28 69-94. · Zbl 1332.62144 · doi:10.1214/12-STS401
[66] Lindsten, F., Jordan, M. I. and Schön, T. B. (2014). Particle Gibbs with ancestor sampling. J. Mach. Learn. Res. 15 2145-2184. · Zbl 1319.60151
[67] Liu, J. and West, M. (2001). Combined parameter and state estimation in simulation-based filtering. In Sequential Monte Carlo Methods in Practice . Springer, New York. · Zbl 1056.93583 · doi:10.1007/978-1-4757-3437-9_10
[68] Liu, J. S. (2001). Monte Carlo Strategies in Scientific Computing . Springer, New York. · Zbl 0991.65001
[69] Liu, J. S. and Chen, R. (1998). Sequential Monte Carlo methods for dynamic systems. J. Amer. Statist. Assoc. 93 1032-1044. · Zbl 1064.65500 · doi:10.2307/2669847
[70] Lopes, H. F., Carvalho, C. M., Johannes, M. S. and Polson, N. G. (2011). Particle learning for sequential Bayesian computation. In Bayesian Statistics 9 (J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D., Heckerman A. F. M. Smith and M., West, eds.). Oxford Univ. Press, Oxford.
[71] Lopes, H. F. and Tsay, R. S. (2011). Particle filters and Bayesian inference in financial econometrics. J. Forecast. 30 168-209. · Zbl 1217.91146 · doi:10.1002/for.1195
[72] Malik, S. and Pitt, M. K. (2011). Particle filters for continuous likelihood evaluation and maximisation. J. Econometrics 165 190-209. · Zbl 1441.62807 · doi:10.1016/j.jeconom.2011.07.006
[73] Nemeth, C., Fearnhead, P. and Mihaylova, L. (2013). Particle approximations of the score and observed information matrix for parameter estimation in state space models with linear computational cost. Preprint. Available at . arXiv:1306.0735
[74] Olsson, J., Cappé, O., Douc, R. and Moulines, E. (2008). Sequential Monte Carlo smoothing with application to parameter estimation in nonlinear state space models. Bernoulli 14 155-179. · Zbl 1155.62055 · doi:10.3150/07-BEJ6150
[75] Olsson, J. and Westerborn, J. (2014). Efficient particle-based online smoothing in general hidden Markov models: The PaRIS algorithm. Preprint. Available at . arXiv:1412.7550 · Zbl 1392.62252
[76] Oudjane, N. and Rubenthaler, S. (2005). Stability and uniform particle approximation of nonlinear filters in case of non ergodic signals. Stoch. Anal. Appl. 23 421-448. · Zbl 1140.93485 · doi:10.1081/SAP-200056643
[77] Paninski, L., Ahmadian, Y., Ferreira, D. G., Koyama, S., Rad, K. R., Vidne, M., Vogelstein, J. and Wu, W. (2010). A new look at state-space models for neural data. J. Comput. Neurosci. 29 107-126. · doi:10.1007/s10827-009-0179-x
[78] Pitt, M. K. and Shephard, N. (1999). Filtering via simulation: Auxiliary particle filters. J. Amer. Statist. Assoc. 94 590-599. · Zbl 1072.62639 · doi:10.2307/2670179
[79] Pitt, M. K., Silva, R. d. S., Giordani, P. and Kohn, R. (2012). On some properties of Markov chain Monte Carlo simulation methods based on the particle filter. J. Econometrics 171 134-151. · Zbl 1443.62499 · doi:10.1016/j.jeconom.2012.06.004
[80] Polson, N. G., Stroud, J. R. and Müller, P. (2008). Practical filtering with sequential parameter learning. J. R. Stat. Soc. Ser. B. Stat. Methodol. 70 413-428. · Zbl 1351.62177 · doi:10.1111/j.1467-9868.2007.00642.x
[81] Poyiadjis, G., Doucet, A. and Singh, S. S. (2011). Particle approximations of the score and observed information matrix in state space models with application to parameter estimation. Biometrika 98 65-80. · Zbl 1214.62093 · doi:10.1093/biomet/asq062
[82] Schön, T. B., Wills, A. and Ninness, B. (2011). System identification of nonlinear state-space models. Automatica J. IFAC 47 39-49. · Zbl 1209.93155 · doi:10.1016/j.automatica.2010.10.013
[83] Sherlock, C., Thiery, A. H., Roberts, G. O. and Rosenthal, J. S. (2015). On the efficiency of pseudo-marginal random walk Metropolis algorithms. Ann. Statist. 43 238-275. · Zbl 1326.65015 · doi:10.1214/14-AOS1278
[84] Storvik, G. (2002). Particle filters in state space models with the presence of unknown static parameters. IEEE Trans. Signal Process. 50 281-289.
[85] Taghavi, E., Lindsten, F., Svensson, L. and Schön, T. B. (2013). Adaptive stopping for fast particle smoothing. In Proc. IEEE ICASSP 6293-6297. Vancouver, BC.
[86] Vercauteren, T., Toledo, A. and Wang, X. (2005). Online Bayesian estimation of hidden Markov models with unknown transition matrix and applications to IEEE 802.11 networks. In Proc. IEEE ICASSP , Vol. IV 13-16. Philadelphia, PA.
[87] West, M. and Harrison, J. (1997). Bayesian Forecasting and Dynamic Models , 2nd ed. Springer, New York. · Zbl 0871.62026 · doi:10.1007/b98971
[88] Westerborn, J. and Olsson, J. (2014). Efficient particle-based online smoothing in general hidden Markov models. In Proc. IEEE ICASSP 8003-8007. Florence. · Zbl 1392.62252
[89] Whiteley, N. (2010). Discussion of Particle Markov chain Monte Carlo methods. J. Royal Stat. Soc. 72 306-307. · doi:10.1111/j.1467-9868.2009.00736.x
[90] Whiteley, N. (2013). Stability properties of some particle filters. Ann. Appl. Probab. 23 2500-2537. · Zbl 1296.60098 · doi:10.1214/12-AAP909
[91] Whiteley, N., Andrieu, C. and Doucet, A. (2010). Efficient Bayesian inference for switching state-space models using discrete particle Markov chain Monte Carlo methods. Preprint. Available at . arXiv:1011.2437 · doi:10.1111/j.1467-9868.2009.00736.x
[92] Whiteley, N. and Lee, A. (2014). Twisted particle filters. Ann. Statist. 42 115-141. · Zbl 1302.60139 · doi:10.1214/13-AOS1167
[93] Wilkinson, D. J. (2012). Stochastic Modelling for Systems Biology , 2nd ed. CRC Press, Boca Raton, FL. · Zbl 1300.92004
[94] Yildirim, S., Singh, S. S. and Doucet, A. (2013). An online expectation-maximization algorithm for changepoint models. J. Comput. Graph. Statist. 22 906-926. · doi:10.1080/10618600.2012.674653
[95] Yuan, Y.-x. (2008). Step-sizes for the gradient method. In Third International Congress of Chinese Mathematicians. Part 1, 2. AMS/IP Stud. Adv. Math. , 42, Pt. 1 2 785-796. Amer. Math. Soc., Providence, RI. · Zbl 1172.90509
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.