×

Performance analysis of local ensemble Kalman filter. (English) Zbl 1394.60038

Summary: Ensemble Kalman filter (EnKF) is an important data assimilation method for high-dimensional geophysical systems. Efficient implementation of EnKF in practice often involves the localization technique, which updates each component using only information within a local radius. This paper rigorously analyzes the local EnKF (LEnKF) for linear systems and shows that the filter error can be dominated by the ensemble covariance, as long as (1) the sample size exceeds the logarithmic of state dimension and a constant that depends only on the local radius; (2) the forecast covariance matrix admits a stable localized structure. In particular, this indicates that with small system and observation noises, the filter error will be accurate in long time even if the initialization is not. The analysis also reveals an intrinsic inconsistency caused by the localization technique, and a stable localized structure is necessary to control this inconsistency. While this structure is usually taken for granted for the operation of LEnKF, it can also be rigorously proved for linear systems with sparse local observations and weak local interactions. These theoretical results are also validated by numerical implementation of LEnKF on a simple stochastic turbulence in two dynamical regimes.

MSC:

60G35 Signal detection and filtering (aspects of stochastic processes)

Software:

EnKF
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Anderson, JL, An ensemble adjustment Kalman filter for data assimilation, Mon. Weather Rev., 129, 2884-2903, (2001)
[2] Bergemann, K; Reich, S, A localization technique for ensemble Kalman filters, Q. J. R. Meteorol. Soc., 136, 701-707, (2010)
[3] Bickel, PJ; Levina, E, Regularized estimation of large covariance matrices, Ann. Stat., 36, 199-227, (2008) · Zbl 1132.62040
[4] Bickel, PJ; Lindner, M, Approximating the inverse of banded matrices by banded matrices with applications to probability and statistics, Theory Probab. Appl., 56, 1-20, (2012) · Zbl 1241.60017
[5] Bishop, A.N., Del Moral, P., Pathiraja, S.D.: Perturbations and projections of Kalman-Bucy semigroups motivated by methods in data assimilation. arXiv:1701.05978 (2017). Accepted by Stochastic Process and their Applications · Zbl 1407.62338
[6] Burgers, G; Leeuwen, PJ; Evensen, G, Analysis scheme in the ensemble Kalman filter, Mon. Weather Rev., 126, 1719-1724, (1998)
[7] Chernov, A., Hoel, H., Law, K.J., Nobile, F., Tembine, H.: Multilevel Ensemble Kalman Filtering for Spatially Extended Models. arxiv:1608.08558 (2006) · Zbl 1370.37009
[8] Chevrotiere, MDL; Harlim, J, A data-driven method for improving the correlation estimation in serial ensemble Kalman filters, Mon. Weather Rev., 145, 985-1001, (2017)
[9] Evensen, G, The ensemble Kalman filter: theoretical formulation and practical implementation, Ocean Dyn., 53, 343-367, (2003)
[10] Furrer, R; Bengtsson, T, Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants, J. Multivar. Anal., 98, 227-255, (2007) · Zbl 1105.62091
[11] Gaspari, G; Cohn, SE, Construction of correlation functions in two and three dimensions, Q. J. R. Meteorol. Soc., 125, 723-757, (1999)
[12] Hamill, TM; Whitaker, C; Snyder, C, Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter, Mon. Weather Rev., 129, 2776-2790, (2001)
[13] Hoel, H; Law, KJ; Tempone, R, Multilevel ensemble Kalman filtering, SIAM J. Numer. Anal., 54, 1813-1839, (2016) · Zbl 1401.65008
[14] Houtekamer, PL; Mitchell, HL, Data assimilation using an ensemble Kalman filter technique, Mon. Weather Rev., 126, 796-811, (1998)
[15] Hunt, BR; Kostelich, EJ; Szunyogh, I, Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter, Phys. D, 230, 112-126, (2007) · Zbl 1115.62030
[16] Janjić, T; Nerger, L; Albertlla, A; Schröter, J; Skachoko, S, On domain localization in ensemble-based Kalman filter algorithms, Mon. Weather Rev., 139, 2046-2060, (2011)
[17] Kalnay, E.: Atmospheric Modeling, Data Assimilation, and Predictability. Cambridge University Press, Cambridge (2003)
[18] Kelly, D.T.B., Stuart, A.M.: Ergodicity and Accuracy of Optimal Particle Filters for Bayesian Data Assimilation. arXiv:1611.08761 (2017)
[19] Kelly, DTB; Law, KJ; Stuart, AM, Well-posedness and accuracy of the ensemble Kalman filter in discrete and continuous time, Nonlinearity, 27, 2579-2603, (2014) · Zbl 1305.62323
[20] Kelly, D; Majda, AJ; Tong, XT, Concrete ensemble Kalman filters with rigorous catastrophic filter divergence, Proc. Natl. Acad. Sci., 112, 10589-10594, (2016)
[21] Künsch, HR; Robert, S, Localizing the ensemble Kalman particle filter, Tellus A Dyn. Meteorol. Oceanogr., 69, 1282016, (2017)
[22] Kwiatkowski, E; Mandel, J, Convergence of the square root ensemble Kalman filter in the large ensemble limit, SIAM/ASA J. Uncertain. Quantif., 3, 1-17, (2015) · Zbl 1329.60083
[23] Law, KJ; Tembine, H; Tempone, R, Deterministic mean-field ensemble Kalman filtering, SIAM J. Sci. Comput., 38, a1251-a1279, (2016) · Zbl 1351.60047
[24] Le Gland, F., Monbet, V., Tran, V.D.: Large sample asymptotics for the ensemble Kalman filter. In: Volume Oxford Handbook of Nonlinear Filtering, pp. 598-631. Oxford University Press, Oxford (2011) · Zbl 1225.93108
[25] Li, H; Kalnay, E; Miyoshi, T, Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter, Q. J. R. Meteorol. Soc., 135, 523-533, (2009)
[26] Liptser, R.S., Shiryaev, A.N.: Statistics of random processes. In: I, II, Volume 5 of Applications of Mathematics. Springer, Berlin (2001) · Zbl 1351.60047
[27] Lorenc, AC, The potential of the ensemble Kalman filter for NWP—a comparison with 4D-var, Q. J. R. Meteorol. Soc., 129, 3183-3203, (2003)
[28] Majda, A.J., Harlim, J.: Filtering Complex Turbulent Systems. Cambridge University Press, Cambridge (2012) · Zbl 1250.93002
[29] Majda, AJ; Lee, Y, State estimation and prediction using clustered particle filters, Proc. Natl. Acad. Sci., 113, 14609-14614, (2016)
[30] Majda, A.J., Tong, X.T.: Rigorous accuracy and robustness analysis for two-scale reduced random Kalman filters in high dimensions. arXiv:1606.09087 (2016) · Zbl 1305.62323
[31] Majda, A.J., Tong, X.T.: Performance of ensemble Kalman filters in large dimensions. Commun. Pure Appl. Math. arXiv:1606.09321 (2017) · Zbl 1390.93799
[32] Mandel, J.: Efficient Implementation of the Ensemble Kalman Filter, Technical report. UCDHSC/CCM Report No. 231, University of Colorado at Denver and Health Sciences Center (2006)
[33] Mandel, J; Cobb, L; Beezley, JD, On the convergence of the ensemble Kalman filter, Appl. Math., 56, 533-541, (2011) · Zbl 1248.62164
[34] Miyoshi, T; Yamane, S, Local ensemble transform Kalman filtering with an AGCM at a T159/L48 resolution, Mon. Weather Rev., 135, 3841-3861, (2007)
[35] Nerger, L, On serial observation processing in localized ensemble Kalman filters, Mon. Weather Rev., 143, 1554-1567, (2015)
[36] Nerger, L; Janjić, T; Schröter, J; Hiller, W, A regulated localization scheme for ensemble-based Kalman filters, Q. J. R. Meteorol. Soc., 138, 802-812, (2012)
[37] Poterjoy, J, A localized particle filter for high-dimensional nonlinear systems, Mon. Weather Rev., 144, 59-76, (2016)
[38] Rebeschini, P; Handel, R, Can local particle filters beat the curse of dimensionality?, Ann. Appl. Probab., 25, 2809-2866, (2015) · Zbl 1325.60058
[39] Rudelson, M; Vershynin, R, Hanson-wright inequality and sub-Gaussian concentration, Electron. Commun. Probab., 18, 1-9, (2013) · Zbl 1329.60056
[40] Sanz-Alonso, D; Stuart, AM, Long time asymptotics of the filtering distribution for partially observed chaotic dynamical systems, SIAM/ASA J. Uncertain. Quantif., 3, 1200-1220, (2015) · Zbl 1337.93094
[41] Snyder, C; Bengtsson, T; Bickel, PJ, Obstacles to high-dimensional particle filtering, Mon. Weather Rev., 136, 4629-4640, (2008)
[42] Tong, XT; Majda, AJ; Kelly, D, Nonlinear stability and ergodicity of ensemble based Kalman filters, Nonlinearity, 29, 657-691, (2016) · Zbl 1334.60060
[43] Tong, XT; Majda, AJ; Kelly, D, Nonlinear stability of the ensemble Kalman filter with adaptive covariance inflation, Commun. Math. Sci., 14, 1283-1313, (2016) · Zbl 1370.37009
[44] Leeuwen, PJ, Particle filtering in geophysical systems, Mon. Weather Rev., 137, 4089-4114, (2009)
[45] Vershynin, R; Eldar, Y (ed.); Kutyniok, G (ed.), Introduction to the non-asymptotic analysis of random matrices, 210-268, (2011), Cambridge
[46] Whitaker, JS; Hamill, TM, Ensemble data assimilation without perturbed observations, Mon. Weather Rev., 130, 1913-1924, (2002)
[47] Whitaker, JS; Hamill, TM; Wei, X; Song, Y; Toth, Z, Ensemble data assimilation with the NCEP global forecast system, Mon. Weather Rev., 136, 463-482, (2008)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.