Validity of the standard cross-correlation test for model structure validation. (English) Zbl 1283.93291

Douma, Sippe G.; Bombois, Xavier; Van den Hof, Paul M. J.

Automatica 44, No. 5, 1285-1294 (2008).

Summary: In the standard prediction error framework of system identification, statistical properties of estimated models are typically derived under the assumption that the true system is in the model class. The standard model structure validation test for plant models is the sample cross-correlation test between the residuals of the model and the input. It turns out that the standard test itself is valid only under exactly those assumptions it is meant to verify, i.e. the system is in the model class. It is shown that for reliable results of the validation test a vector-valued test is required and that accurate noise modelling is indispensable for reliable model structure validation. This shows the limitation of separate validation of plant and noise model structures. Improvements of the test are presented, and it is motivated by the fact that reserving data only to be used for model validation is not efficient.

MSC:

93E12	Identification in stochastic control theory
93C41	Control/observation systems with incomplete information
93C05	Linear systems in control theory

Keywords:

model validation; system identification; undermodelling; model uncertainty; statistical tests

Software:

System Identification Toolbox; ARMASA

References:

[1]	Broersen, P. M.T., Automatic spectral analysis with time series models, IEEE Transactions on Instrumentation and Measurement, 51, 2, 211-216 (2002)
[2]	Broersen, P. M.T., (ARMASA toolbox in matlab (2003), The Mathworks, Inc)
[3]	De Vries, D. K.; Van den Hof, P. M.J., Quantification of uncertainty in transfer function estimation: A mixed probabilistic-worst-case approach, Automatica, 31, 4, 543-557 (1995) · Zbl 0822.93022
[9]	Goodwin, G. C.; Gevers, M.; Ninness, B. M., Quantifying the error in estimated transfer functions with application to model order selection, IEEE Transactions on Automatic Control, 37, 913-928 (1992) · Zbl 0767.93022
[10]	Hakvoort, R. G.; Van den Hof, P. M.J., Identification of model error bounds in \(\ell_1\) and \(H_\infty \)-norm, (Smith, R. S.; Dahleh, M. A., The modeling of uncertainty in control systems. The modeling of uncertainty in control systems, Lecture notes in control and information sciences, Vol. 192 (1994), Springer Verlag), 139-155 · Zbl 0793.93022
[11]	Hakvoort, R. G.; Van den Hof, P. M.J., Identification of probabilistic uncertainty regions by explicit evaluation of bias and variance errors, IEEE Transactions on Automatic Control, 42, 11, 1516-1528 (1997) · Zbl 0915.93067
[12]	(Heuberger, P. S.C.; Van den Hof, P. M.J.; Wahlberg, B., Modelling and identification with rational orthogonal basis functions (2005), Springer Verlag: Springer Verlag London, UK)
[13]	Hjalmarsson, H., From experiment design to closed-loop control, Automatica, 41, 393-438 (2005) · Zbl 1079.93016
[14]	Hjalmarsson, H.; Ljung, L., Estimating model variance in the case of undermodeling, IEEE Transactions on Automatic Control, 37, 1004-1008 (1992) · Zbl 0767.93003
[15]	Ljung, L., Model validation and model error modeling, (Wittenmark, B.; Rantzer, A., Proceedings of the Åström symposium on control (1999)), 15-42
[16]	Ljung, L., System identification: Theory for the user (1999), Prentice-Hall: Prentice-Hall Englewood Cliffs, NJ, USA
[17]	Ljung, L., System identification toolbox in matlab (2003), The Mathworks, Inc
[18]	Ljung, L.; Guo, L., The role of model validation for assessing the size of the unmodelled dynamics, IEEE Transactions on Automatic Control, 42, 9, 1230-1239 (1997) · Zbl 0889.93018
[19]	Poolla, K.; Khargonekar, P. P.; Tikku, A.; Krause, J.; Nagpal, K., A time-domain approach to model validation, IEEE Transactions on Automatic Control, 39, 951-959 (1994) · Zbl 0813.93011
[20]	Schoukens, J.; Pintelon, R.; Rolain, Y., Box-Jenkins alike identification using non-parametric noise models, Automatica, 40, 2083-2089 (2004) · Zbl 1077.93052
[22]	Smith, R.; Doyle, J., Model validation: A connection between robust control and identification, IEEE Transactions on Automatic Control, 37, 942-952 (1992) · Zbl 0767.93020
[23]	Söderström, T.; Stoica, P., System Identification (1989), Prentice-Hall: Prentice-Hall Hemel Hempstead, UK · Zbl 0714.93056
[24]	Tjärnström, F.; Ljung, L., Using the bootstrap to estimate the variance in the case of undermodeling, IEEE Transactions on Automatic Control, 47, 2, 395-398 (2002) · Zbl 1364.93130