×

zbMATH — the first resource for mathematics

Maximum entropy properties of discrete-time first-order stable spline kernel. (English) Zbl 1335.93131
Summary: The first order Stable Spline (SS-1) kernel (also known as the Tuned-Correlated (TC) kernel) is used extensively in regularized system identification, where the impulse response is modeled as a zero-mean Gaussian process whose covariance function is given by well designed and tuned kernels. In this paper, we discuss the maximum entropy properties of this kernel. In particular, we formulate the exact maximum entropy problem solved by the SS-1 kernel without Gaussian and uniform sampling assumptions. Under general sampling assumption, we also derive the special structure of the SS-1 kernel (e.g. its tridiagonal inverse and factorization have closed form expression), also giving to it a maximum entropy covariance completion interpretation.

MSC:
93E12 Identification in stochastic control theory
93C57 Sampled-data control/observation systems
37M25 Computational methods for ergodic theory (approximation of invariant measures, computation of Lyapunov exponents, entropy, etc.)
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Ardeshiri, T., & Chen, T. (2015). Maximum entropy property of discrete-time first-order stable spline kernel. In the 40th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2015, Brisbane, Australia (pp. 3676-3680).
[2] Carli, F.P. (2014). On the maximum entropy property of the first-order stable spline kernel and its implications. In IEEE multi-conference on systems and control, Nice/Antibes, France (pp. 409-414).
[3] Carli, F.P., Chen, T., & Ljung, L. (2014). Maximum entropy kernels for system identification. arXiv:1411.5620. · Zbl 1366.93669
[4] Chen, T.; Andersen, M. S.; Ljung, L.; Chiuso, A.; Pillonetto, G., System identification via sparse multiple kernel-based regularization using sequential convex optimization techniques, IEEE Transactions on Automatic Control, 11, 2933-2945, (2014) · Zbl 1360.93720
[5] Chen, T.; Ljung, L., Implementation of algorithms for tuning parameters in regularized least squares problems in system identification, Automatica, 49, 2213-2220, (2013) · Zbl 1364.93825
[6] Chen, T., & Ljung, L. (2015). On kernel structure for regularized system identification (ii): a system theory perspective. In Proceedings of the IFAC symposium on system identification, Beijing, China (pp. 1041-1046).
[7] Chen, T.; Ohlsson, H.; Ljung, L., On the estimation of transfer functions, regularizations and Gaussian processes — revisited, Automatica, 48, 1525-1535, (2012) · Zbl 1269.93126
[8] Chiuso, A., Chen, T., Ljung, L., & Pillonetto, G. (2014). On the design of multiple kernels for nonparametric linear system identification. In Proceedings of the IEEE conference on decision and control, Los Angeles, CA (pp. 3346-3351).
[9] Cover, T. M.; Thomas, J. A., Elements of information theory, (2012), John Wiley & Sons
[10] Dempster, A. P., Covariance selection, Biometrics, 28, 1, 157-175, (1972)
[11] Gohberg, I.; Goldberg, S.; Kaashoek, A., (Classes of linear operators, Operator theory, advances and applications, (1993), Birkhäuser Verlag)
[12] Jaynes, E. T, On the rationale of maximum-entropy methods, Proceedings of the IEEE, 70, 9, 939-952, (1982)
[13] Ljung, L., System identification — theory for the user, (1999), Prentice-Hall Upper Saddle River, N.J
[14] Pillonetto, G.; Chiuso, A.; De Nicolao, G., Prediction error identification of linear systems: a nonparametric Gaussian regression approach, Automatica, 47, 2, 291-305, (2011) · Zbl 1207.93110
[15] Pillonetto, G.; De Nicolao, G., A new kernel-based approach for linear system identification, Automatica, 46, 1, 81-93, (2010) · Zbl 1214.93116
[16] Pillonetto, G., & De Nicolao, G. (2011). Kernel selection in linear system identification. Part I: A Gaussian process perspective. In Proc. 50th IEEE Conference on Decision and Control, Orlando, Florida (pp. 4318-4325).
[17] Pillonetto, G.; Dinuzzo, F.; Chen, T.; De Nicolao, G.; Ljung, L., Kernel methods in system identification, machine learning and function estimation: A survey, Automatica, 50, 3, 657-682, (2014) · Zbl 1298.93342
[18] Rasmussen, C. E.; Williams, C. K.I., Gaussian processes for machine learning, (2006), MIT Press Cambridge, MA · Zbl 1177.68165
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.