# zbMATH — the first resource for mathematics

A new kernel-based approach for linear system identification. (English) Zbl 1214.93116
Summary: This paper describes a new kernel-based approach for linear system identification of stable systems. We model the impulse response as the realization of a Gaussian process whose statistics, differently from previously adopted priors, include information not only on smoothness but also on BIBO-stability. The associated autocovariance defines what we call a stable spline kernel. The corresponding minimum variance estimate belongs to a reproducing kernel Hilbert space which is spectrally characterized. Compared to parametric identification techniques, the impulse response of the system is searched within an infinite-dimensional space, dense in the space of continuous functions. Overparametrization is avoided by tuning few hyperparameters via marginal likelihood maximization. The proposed approach may prove particularly useful in the context of robust identification in order to obtain reduced order models by exploiting a two-step procedure that projects the nonparametric estimate onto the space of nominal models. The continuous-time derivation immediately extends to the discrete-time case. On several continuous- and discrete-time benchmarks taken from the literature the proposed approach is very efficient compared with the existing parametric and nonparametric techniques.

##### MSC:
 93E12 Identification in stochastic control theory 93C05 Linear systems in control theory 93E10 Estimation and detection in stochastic control theory
##### Software:
System Identification Toolbox
Full Text:
##### References:
 [1] Adams, R.A.; Fournier, J., Sobolev spaces, (2003), Academic Press · Zbl 1098.46001 [2] Anderson, B.D.O.; Moore, J.B., Optimal filtering, (1979), Prentice-Hall Englewood Cliffs, NJ, USA · Zbl 0758.93070 [3] Aronszajn, N., Theory of reproducing kernels, Transactions of the American mathematical society, 68, 337-404, (1950) · Zbl 0037.20701 [4] Barry, D., Nonparametric Bayesian regression, The annals of statistics, 14, 934-953, (1986) · Zbl 0608.62052 [5] Bell, B.M.; Pillonetto, G., Estimating parameters and stochastic functions of one variable using nonlinear measurements models, Inverse problems, 20, 3, 627-646, (2004) · Zbl 1055.62028 [6] Bertero, M., Linear inverse and ill-posed problems, Advances in electronics and electron physics, 75, 1-120, (1989) [7] Burenkov, V.I., Sobolev spaces on domains, (1998), Teubner-Texte zur Mathematik · Zbl 0893.46024 [8] Chiuso, A., Pillonetto, G., & De Nicolao, G. (2008). Subspace identification using predictor estimation via Gaussian regression. In Proceedings of the IEEE conf. on dec. and control. Cancun, Mexico · Zbl 1207.93110 [9] Cucker, F.; Smale, S., On the mathematical foundations of learning, Bulletin of the American mathematical society, 39, 1-49, (2001) · Zbl 0983.68162 [10] Ferrari-Trecate, G.; Williams, C.K.I.; Opper, M., Finite-dimensional approximation of Gaussian processes, () [11] Freedman, D., On the Bernstein-von Mises theorem with infinite-dimensional parameters, The annals of statistics, 27, 1119-1140, (1999) · Zbl 0957.62002 [12] Garulli, A.; Vicino, A.; Zappa, G., Conditional central algorithms for worst-case set-membership identification and filtering, IEEE transaction on automatic control, 45, 1, 14-23, (2000) · Zbl 0971.93072 [13] Giarre’, L.; Milanese, M.; Taragna, M., $$H_\infty$$ identification and model quality evaluation, IEEE transaction on automatic control, 42, 188-199, (1997) · Zbl 0870.93010 [14] Goodwin, G.C.; Braslavsky, J.H.; Seron, M.M., Non-stationary stochastic embedding for transfer function estimation, Automatica, 38, 47-62, (2002) · Zbl 1002.93063 [15] Goodwin, G.C.; Gevers, M.; Ninness, B., Quantifying the error in estimated transfer functions with application to model order selection, IEEE transaction on automatic control, 37, 7, 913-928, (1992) · Zbl 0767.93022 [16] Hakvoort, R.G.; Van den Hof, P.M.J., Identification of probabilistic system uncertainty regions by explicit evaluation of bias and variance errors, IEEE transaction on automatic control, 42, 1516-1528, (1997) · Zbl 0915.93067 [17] Hjalmarsson, H., From experiment design to closed-loop control, Automatica, 41, 393-438, (2005) · Zbl 1079.93016 [18] Hjalmarsson, H.; Gustafsson, F., Composite modeling of transfer functions, IEEE transaction on automatic control, 40, 820-832, (1995) · Zbl 0824.93066 [19] Johansen, T.A., On Tikhonov regularization, bias and variance in nonlinear system identification, Automatica, 33, 441-446, (1997) · Zbl 0873.93024 [20] Kass, R.E.; Raftery, A.E., Bayes factors, Journal of the American statistical association, 90, 773-795, (1995) · Zbl 0846.62028 [21] Lecchini, A.; Gevers, M., Explicit expression of the parameter bias in identification of Laguerre models from step responses, Systems and control letters, 52, 149-165, (2004) · Zbl 1157.93357 [22] Ljung, L. (1999a). Model validation and model error modeling. In Proceedings of the Astrom symposium on control. (pp. 15-42) Studentliteratur, Lund, Sweden [23] Ljung, L., System identification — theory for the user, (1999), Prentice Hall [24] Ljung, L., System identification toolbox for Matlab, v7.2, (2008), The MathWorks, Inc Natick, MA [25] Luenberger, D.G., Linear and nonlinear programming, (1989), Addison Wesley · Zbl 0241.90052 [26] Lukic, M.N.; Beder, J.H., Stochastic processes with sample paths in reproducing kernel Hilbert spaces, Transactions of the American mathematical society, 353, 3945-3969, (2001) · Zbl 0973.60036 [27] Makila, P.M.; Partington, J.R.; Gustafsson, T., Worst-case control-relevant identification, Automatica, 31, 1799-1820, (1995) · Zbl 0846.93022 [28] Milanese, M.; Norton, J.P.; Piet-Lahanier, H.; Walter, E., Bounding approaches to system identification, (1996), Plenum Press New York, NY, USA · Zbl 0845.00024 [29] Milanese, M.; Vicino, A., Optimal estimation theory for dynamic systems with set membership uncertainty: an overview, Automatica, 27, 6, 997-1009, (1991) · Zbl 0737.62088 [30] Neve, M.; De Nicolao, G.; Marchesi, L., Nonparametric identification of population models via Gaussian processes, Automatica, 97, 7, 1134-1144, (2007) · Zbl 1123.93319 [31] Pillonetto, G.; Bell, B.M., Bayes and empirical Bayes semi-blind deconvolution using eigenfunctions of a prior covariance, Automatica, 43, 10, 1698-1712, (2007) · Zbl 1119.93064 [32] Pillonetto, G., Chiuso, A., & De Nicolao, G. (2008). Predictor estimation via Gaussian regression. In Proceedings of the IEEE conf. on dec. and control, Cancun, Mexico · Zbl 1207.93110 [33] Pillonetto, G.; Saccomani, M.P., Input estimation in nonlinear dynamic systems using differential algebra concepts, Automatica, 42, 2117-2129, (2006) · Zbl 1104.93020 [34] Poggio, T.; Girosi, F., Networks and the best approximation property, Biological cybernetics, 63, 169-176, (1990) · Zbl 0714.94029 [35] Poggio, T.; Girosi, F., Networks for approximation and learning, Proceedings of the IEEE, 78, 1481-1497, (1990) · Zbl 1226.92005 [36] Rasmussen, C.E.; Williams, C.K.I., Gaussian processes for machine learning, (2006), The MIT Press [37] Reinelt, W.; Garulli, A.; Ljung, L., Comparing different approaches to model error modeling in robust identification, Automatica, 38, 5, 787-803, (2002) · Zbl 1004.93014 [38] Runge, C., Uber empirische funktionen und die interpolation zwischen aquidistanten ordinaten, Zeitschrift für Mathematik und physik, 46, 224-243, (1901) · JFM 32.0272.02 [39] Smale, S.; Zhou, D.X., Learning theory estimates via integral operators and their approximations, Constructive approximation, 26, 153-172, (2007) · Zbl 1127.68088 [40] Smola, A.J.; Schölkopf, B.M., Bayesian kernel methods, (), 65-117 · Zbl 1019.68095 [41] Söderström, T.; Stoica, P., System identification, (1989), Prentice Hall · Zbl 0714.93056 [42] Stenman, A., & Tjarnstrom, F. (2000) A nonparametric approach to model error modeling. In Proceedings of the 12th IFAC symposium on system identification (pp. 157-162) Santa Barbara, USA [43] Sun, H., Mercer theorem for RKHS on noncompact sets, Journal of complexity, 21, 337-349, (2005) · Zbl 1094.46021 [44] Suykens, J.R.; Van Gestel, T.; De Brabanter, J.; De Moor, B.; Vandewalle, J., Least squares support vector machines, (2002), World Scientific Singapore · Zbl 1017.93004 [45] Thompson, J.R.; Tapia, R.A., Nonparametric function estimation, modelling and simulation, (1990), SIAM Philadelphia, PA · Zbl 0709.62038 [46] Tikhonov, A.N.; Arsenin, V.Y., Solutions of ill-posed problems, (1977), Winston/Wiley Washington, DC · Zbl 0354.65028 [47] Wahba, G., Practical approximate solutions to linear operator equations when the data are noisy, SIAM journal on numerical analysis, 14, 651-667, (1977) · Zbl 0402.65032 [48] Wahba, G., Spline models for observational data, (1990), SIAM Philadelphia · Zbl 0813.62001 [49] Wahba, G. (1998). Support vector machines, reproducing kernel Hilbert spaces and randomized GACV. Technical report. Department of Statistics, University of Wisconsin [50] Wahlberg, B.; Ljung, L., Design variables for bias distribution in transfer function estimation, IEEE transaction on automatic control, 31, 2, 134-144, (1986) · Zbl 0582.93065 [51] Williams, C.K.I.; Rasmussen, C.E., Gaussian processes for regression, (), 514-520 [52] Zhu, H.; Rohwer, R., Bayesian regression filters and the issue of priors, Neural computing and applications, 4, 130-142, (1995) [53] Zhu, H.; Williams, C.K.I.; Rohwer, R.; Morciniec, M., Gaussian regression and optimal finite dimensional linear models, ()
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.