Maximum likelihood least squares identification for systems with autoregressive moving average noise. (English) Zbl 1242.62105

Wang, Wei; Ding, Feng; Dai, Jiyang

Appl. Math. Modelling 36, No. 5, 1842-1853 (2012).

Summary: Maximum likelihood methods are important for system modeling and parameter estimation. This paper derives a recursive maximum likelihood least squares identification algorithm for systems with autoregressive moving average noises, based on the maximum likelihood principle. In this derivation, we prove that the maximum of the likelihood function is equivalent to minimizing the least squares cost function. The proposed algorithm is different from the corresponding generalized extended least squares algorithm. A simulation test shows that the proposed algorithm has a higher estimation accuracy than the recursive generalized extended least squares algorithm.

MSC:

62M10	Time series, auto-correlation, regression, etc. in statistics (GARCH)
62F10	Point estimation
65C60	Computational problems in statistics (MSC2010)

Keywords:

recursive identification; CARARMA system

References:

[1]	Xiao, Y.S.; Ding, F.; Zhou, Y.; Li, M.; Dai, J.Y., On consistency of recursive least squares identification algorithms for controlled auto-regression models, Applied mathematical modelling, 32, 11, 2207-2215, (2008) · Zbl 1156.93411
[2]	Zhang, Y.; Cui, G.M., Bias compensation methods for stochastic systems with colored noise, Applied mathematical modelling, 35, 4, 1709-1716, (2011) · Zbl 1217.93163
[3]	Ljung, L., System identification: theory for the user, (1999), Prentice-Hall Englewood Cliffs, New Jersey
[4]	Kayahan, B.; Stengos, T., Testing the capital asset pricing model with local maximum likelihood methods, Mathematical and computer modelling, 46, 1-2, 138-150, (2007) · Zbl 1142.91461
[5]	Lundahl, T.; Ohley, W.J.; Kay, S.M.; Siffert, R., Fractional Brownian motion: a maximum likelihood estimator and its application to image texture, IEEE transactions on medical imaging, 5, 3, 152-161, (2007)
[6]	Bahl, L.R.; Jelinek, F.; Mercer, R.L., A maximum likelihood approach to continuous speech recognition, IEEE transactions on pattern analysis and machine intelligence, 5, 2, 179-190, (2009)
[7]	Kyung, M.; Ghosh, S.K., Maximum likelihood estimation for directional conditionally autoregressive models, Journal of statistical planning and inference, 140, 11, 3160-3179, (2010) · Zbl 1204.62164
[8]	Södersöm, T.; Hong, M.; Schoukens, J.; Pintelon, R., Accuracy analysis of time domain maximum likelihood method and sample maximum likelihood method for errors-in-variables and output error identification, Automatica, 46, 4, 721-727, (2010) · Zbl 1193.93180
[9]	Agüero, J.C.; Yuz, J.I.; Goodwin, G.C.; Delgado, R.A., On the equivalence of time and frequency domain maximum likelihood estimation, Automatica, 46, 2, 260-270, (2010) · Zbl 1205.93141
[10]	Ding, F.; Ding, J., Least squares parameter estimation with irregularly missing data, International journal of adaptive control and signal processing, 24, 7, 540-553, (2010) · Zbl 1200.93130
[11]	Han, L.L.; Sheng, J.; Ding, F.; Shi, Y., Auxiliary model identification method for multirate multi-input systems based on least squares, Mathematical and computer modelling, 50, 7-8, 1100-1106, (2009) · Zbl 1185.93139
[12]	Wang, D.Q.; Chu, Y.Y.; Ding, F., Auxiliary model-based RELS and MI-ELS algorithms for Hammerstein OEMA systems, Computers & mathematics with applications, 59, 9, 3092-3098, (2010) · Zbl 1193.93170
[13]	Liu, Y.J.; Wang, D.Q.; Ding, F., Least-squares based iterative algorithms for identifying box – jenkins models with finite measurement data, Digital signal processing, 20, 5, 1458-1467, (2010)
[14]	Ding, F.; Liu, P.X.; Liu, G., Gradient based and least-squares based iterative identification methods for OE and OEMA systems, Digital signal processing, 20, 3, 664-677, (2010)
[15]	Liu, X.G.; Lu, J., Least squares based iterative identification for a class of multirate systems, Automatica, 46, 3, 549-554, (2010) · Zbl 1194.93079
[16]	Wang, D.Q.; Yang, G.W.; Ding, R.F., Gradient-based iterative parameter estimation for box – jenkins systems, Computers & mathematics with applications, 60, 5, 1200-1208, (2010) · Zbl 1201.94046
[17]	Ding, J.; Ding, F., The residual based extended least squares identification method for dual-rate systems, Computers & mathematics with applications, 56, 6, 1479-1487, (2008) · Zbl 1155.93435
[18]	Ding, F.; Chen, T., Identification of Hammerstein nonlinear ARMAX systems, Automatica, 41, 9, 1479-1489, (2005) · Zbl 1086.93063
[19]	Han, H.Q.; Xie, L.; Ding, F.; Liu, X.G., Hierarchical least squares based iterative identification for multivariable systems with moving average noises, Mathematical and computer modelling, 51, 9-10, 1213-1220, (2010) · Zbl 1198.93216
[20]	Zhang, Z.N.; Ding, F.; Liu, X.G., Hierarchical gradient based iterative parameter estimation algorithm for multivariable output error moving average systems, Computers & mathematics with applications, 61, 3, 672-682, (2011) · Zbl 1217.15022
[21]	Wang, D.Q.; Ding, F., Extended stochastic gradient identification algorithms for hammerstein – wiener ARMAX systems, Computers & mathematics with applications, 56, 12, 3157-3164, (2008) · Zbl 1165.65308
[22]	Chen, J.; Ding, F., Modified stochastic gradient algorithms with fast convergence rates, Journal of vibration and control, 17, 9, 1281-1286, (2011) · Zbl 1271.93043
[23]	Xiao, Y.S.; Zhang, Y.; Ding, J.; Dai, J.Y., The residual based interactive least squares algorithms and simulation studies, Computers & mathematics with applications, 58, 6, 1190-1197, (2009) · Zbl 1189.62149
[24]	Ding, F.; Liu, G.; Liu, X.P., Partially coupled stochastic gradient identification methods for non-uniformly sampled systems, IEEE transactions on automatic control, 55, 8, 1976-1981, (2010) · Zbl 1368.93121
[25]	Wang, D.Q.; Ding, F., Input – output data filtering based recursive least squares identification for CARARMA systems, Digital signal processing, 20, 4, 991-999, (2010)
[26]	W. Wang, J.H. Li, R.F. Ding, Maximum likelihood identification algorithm for controlled autoregressive autoregressive models, International Journal of Computer Mathematics, doi:10.1080/00207160.2011.598514. · Zbl 1248.93161
[27]	Ding, F.; Liu, P.X.; Liu, G., Identification methods for Hammerstein nonlinear systems, Digital signal processing, 21, 2, 215-238, (2011)
[28]	Wang, D.Q.; Ding, F., Least squares based and gradient based iterative identification for Wiener nonlinear systems, Signal processing, 91, 5, 1182-1189, (2011) · Zbl 1219.94052
[29]	Ding, F.; Shi, Y.; Chen, T., Auxiliary model based least-squares identification methods for Hammerstein output-error systems, Systems & control letters, 56, 5, 373-380, (2007) · Zbl 1130.93055
[30]	Ding, F.; Liu, G.; Liu, X.P., Parameter estimation with scarce measurements, Automatica, 47, 8, 1646-1655, (2011) · Zbl 1232.62043
[31]	Ding, J.; Shi, Y.; Wang, H.G.; Ding, F., A modified stochastic gradient based parameter estimation algorithm for dual-rate sampled-data systems, Digital signal processing, 20, 4, 1238-1247, (2010)
[32]	Ding, F.; Liu, P.X.; Yang, H.Z., Parameter identification and intersample output estimation for dual-rate systems, IEEE transactions on systems, man, and cybernetics, part A: systems and humans, 38, 4, 966-975, (2008)
[33]	Liu, Y.J.; Xie, L.; Ding, F., An auxiliary model based recursive least squares parameter estimation algorithm for non-uniformly sampled multirate systems, Proceedings of the institution of mechanical engineers, part I: journal of systems and control engineering, 223, 4, 445-454, (2009)
[34]	Ding, F.; Qiu, L.; Chen, T., Reconstruction of continuous-time systems from their non-uniformly sampled discrete-time systems, Automatica, 45, 2, 324-332, (2009) · Zbl 1158.93365
[35]	Xie, L.; Yang, H.Z.; Ding, F., Recursive least squares parameter estimation for non-uniformly sampled systems based on the data filtering, Mathematical and computer modelling, 54, 1-2, 315-324, (2011) · Zbl 1225.62120
[36]	Liu, Y.J.; Sheng, J.; Ding, R.F., Convergence of stochastic gradient estimation algorithm for multivariable ARX-like systems, Computers & mathematics with applications, 59, 8, 2615-2627, (2010) · Zbl 1193.60057
[37]	Ding, F.; Chen, T., Performance analysis of multi-innovation gradient type identification methods, Automatica, 43, 1, 1-14, (2007) · Zbl 1140.93488
[38]	Ding, F., Several multi-innovation identification methods, Digital signal processing, 20, 4, 1027-1039, (2010)
[39]	Wang, D.Q.; Ding, F., Performance analysis of the auxiliary models based multi-innovation stochastic gradient estimation algorithm for output error systems, Digital signal processing, 20, 3, 750-762, (2010)
[40]	Han, L.L.; Ding, F., Identification for multirate multi-input systems using the multi-innovation identification theory, Computers & mathematics with applications, 57, 9, 1438-1449, (2009) · Zbl 1186.93076
[41]	Han, L.L.; Ding, F., Multi-innovation stochastic gradient algorithms for multi-input multi-output systems, Digital signal processing, 19, 4, 545-554, (2009)
[42]	Zhang, J.B.; Ding, F.; Shi, Y., Self-tuning control based on multi-innovation stochastic gradient parameter estimation, Systems & control letters, 58, 1, 69-75, (2009) · Zbl 1154.93040
[43]	Ding, F.; Liu, X.P.; Liu, G., Auxiliary model based multi-innovation extended stochastic gradient parameter estimation with colored measurement noises, Signal processing, 89, 10, 1883-1890, (2009) · Zbl 1178.94137
[44]	Liu, Y.J.; Xiao, Y.S.; Zhao, X.L., Multi-innovation stochastic gradient algorithm for multiple-input single-output systems using the auxiliary model, Applied mathematics and computation, 215, 4, 1477-1483, (2009) · Zbl 1177.65095
[45]	Xie, L.; Yang, H.Z.; Ding, F., Modeling and identification for non-uniformly periodically sampled-data systems, IET control theory & applications, 4, 5, 784-794, (2010)
[46]	Ding, F.; Liu, P.X.; Liu, G., Multi-innovation least squares identification for system modeling, IEEE transactions on systems, man, and cybernetics, part B: cybernetics, 40, 3, 767-778, (2010)
[47]	Liu, Y.J.; Yu, L.; Ding, F., Multi-innovation extended stochastic gradient algorithm and its performance analysis, Circuits, systems and signal processing, 29, 4, 649-667, (2010) · Zbl 1196.94026
[48]	Chen, J.; Zhang, Y.; Ding, R.F., Auxiliary model based multi-innovation algorithms for multivariable nonlinear systems, Mathematical and computer modelling, 52, 9-10, 1428-1434, (2010) · Zbl 1205.93142
[49]	Yin, H.H.; Zhu, Z.F.; Ding, F., Model order determination using the Hankel matrix of impulse responses, Applied mathematics letters, 24, 5, 797-802, (2011) · Zbl 1211.93042
[50]	Chen, L.; Li, J.H.; Ding, R.F., Identification of the second-order systems based on the step response, Mathematical and computer modelling, 53, 5-6, 1074-1083, (2011) · Zbl 1217.93043
[51]	Bao, B.; Xu, Y.Q.; Sheng, J.; Ding, R.F., Least squares based iterative parameter estimation algorithm for multivariable controlled ARMA system modelling with finite measurement data, Mathematical and computer modelling, 53, 9-10, 1664-1669, (2011) · Zbl 1219.62133
[52]	Xiang, L.L.; Xie, L.B.; Liao, Y.W.; Ding, R.F., Hierarchical least squares algorithms for single-input multiple-output systems based on the auxiliary model, Mathematical and computer modelling, 52, 5-6, 918-924, (2010) · Zbl 1202.93085
[53]	Han, H.Q.; Song, G.L.; Xiao, Y.S.; Liao, Y.W.; Ding, R.F., Performance analysis of the AM-SG parameter estimation for multivariable systems, Applied mathematics and computation, 217, 12, 5566-5572, (2011) · Zbl 1207.62121