×

On the stability of reproducing kernel Hilbert spaces of discrete-time impulse responses. (English) Zbl 1402.93091

Summary: Reproducing Kernel Hilbert Spaces (RKHSs) have proved themselves to be key tools for the development of powerful machine learning algorithms, the so-called regularized kernel-based approaches. Recently, they have also inspired the design of new linear system identification techniques able to challenge classical parametric prediction error methods. These facts motivate the study of the RKHS theory within the control community. In this note, we focus on the characterization of stable RKHSs, i.e. RKHSs of functions representing stable impulse responses. Related to this, working in an abstract functional analysis framework, C. Carmeli et al. [Anal. Appl., Singap. 4, No. 4, 377–408 (2006; Zbl 1116.46019)] has provided conditions for an RKHS to be contained in the classical Lebesgue spaces \(\mathcal{L}^p\). In particular, we specialize this analysis to the discrete-time case with \(p = 1\). Necessary and sufficient conditions for the stability of an RKHS are worked out by a quite simple proof, more easily accessible to the control community.

MSC:

93B30 System identification
93C55 Discrete-time control/observation systems
93C25 Control/observation systems in abstract spaces
93C05 Linear systems in control theory
93D05 Lyapunov and other classical stabilities (Lagrange, Poisson, \(L^p, l^p\), etc.) in control theory
68T05 Learning and adaptive systems in artificial intelligence

Citations:

Zbl 1116.46019
PDF BibTeX XML Cite
Full Text: DOI

References:

[1] Aravkin, A. Y.; Bell, B. M.; Burke, J. V.; Pillonetto, G., The connection between Bayesian estimation of a Gaussian random field and RKHS, IEEE Transactions on Neural Networks and Learning Systems, 26, 7, 1518-1524, (2015)
[2] Aronszajn, N., Theory of reproducing kernels, Transactions of the American Mathematical Society, 68, 337-404, (1950) · Zbl 0037.20701
[3] Bergman, S., (The Kernel function and conformal mapping, Mathematical surveys and monographs, (1950), AMS)
[4] Bertero, M., Linear inverse and ill-posed problems, Advances in Electronics and Electron Physics, 75, 1-120, (1989)
[5] Carmeli, C.; De Vito, E.; Toigo, A., Vector valued reproducing kernel Hilbert spaces of integrable functions and Mercer theorem, Analysis and Applications, 4, 377-408, (2006) · Zbl 1116.46019
[6] Chen, T.; Andersen, M. S.; Ljung, L.; Chiuso, A.; Pillonetto, G., System identification via sparse multiple kernel-based regularization using sequential convex optimization techniques, IEEE Transactons on Automatic Control, 59, 11, 2933-2945, (2014) · Zbl 1360.93720
[7] Chen, T.; Ohlsson, H.; Ljung, L., On the estimation of transfer functions, regularizations and Gaussian processes - revisited, Automatica, 48, 8, 1525-1535, (2012) · Zbl 1269.93126
[8] Cucker, F.; Smale, S., On the mathematical foundations of learning, Bulletin of the American Mathematical Society, 39, 1-49, (2001) · Zbl 0983.68162
[9] Dinuzzo, F., Kernels for linear time invariant system identification, SIAM Journal on Control and Optimization, 53, 5, 3299-3317, (2015) · Zbl 1329.93049
[10] Drucker, H.; Burges, C. J.C.; Kaufman, L.; Smola, A.; Vapnik, V., Support vector regression machines, (Advances in neural information processing systems, (1997))
[11] Evgeniou, T.; Pontil, M.; Poggio, T., Regularization networks and support vector machines, Advances in Computational Mathematics, 13, 1-150, (2000) · Zbl 0939.68098
[12] Girosi, F. (1997). An equivalence between sparse approximation and support vector machines. Technical report, Cambridge, MA, USA.
[13] Kennedy, R. A.; Sadeghi, P., Hilbert space methods in signal processing, (2013), Cambridge University Press Cambridge, UK
[14] Kimeldorf, G.; Wahba, G., A correspondence between bayesan estimation of stochastic processes and smoothing by splines, The Annals of Mathematical Statistics, 41, 2, 495-502, (1971) · Zbl 0193.45201
[15] Ljung, L., System identification - theory for the user, (1999), Prentice-Hall Upper Saddle River, N.J.
[16] Lukic, M. N.; Beder, J. H., Stochastic processes with sample paths in reproducing kernel Hilbert spaces, Transactions of the American Mathematical Society, 353, 3945-3969, (2001) · Zbl 0973.60036
[17] Megginson, R. E., An introduction to Banach space theory, (1998), Springer · Zbl 0910.46008
[18] Pillonetto, G.; Chiuso, A.; De Nicolao, G., Prediction error identification of linear systems: a nonparametric Gaussian regression approach, Automatica, 47, 2, 291-305, (2011) · Zbl 1207.93110
[19] Pillonetto, G.; De Nicolao, G., A new kernel-based approach for linear system identification, Automatica, 46, 1, 81-93, (2010) · Zbl 1214.93116
[20] Pillonetto, G., Dinuzzo, F., Chen, T., Nicolao, G. D., & Ljung, L. (2014). Kernel methods in system identification, machine learning and function estimation: a survey, 50(3), 657-682. · Zbl 1298.93342
[21] Poggio, T.; Girosi, F., Networks for approximation and learning, (Proceedings of the IEEE, Vol. 78, (1990)), 1481-1497 · Zbl 1226.92005
[22] Schölkopf, B.; Smola, A. J., (Learning with Kernels: Support vector machines, regularization, optimization, and beyond, (Adaptive computation and machine learning), (2001), MIT Press)
[23] Smale, S.; Zhou, D. X., Learning theory estimates via integral operators and their approximations, Constructive Approximation, 26, 153-172, (2007) · Zbl 1127.68088
[24] Söderström, T.; Stoica, P., System identification, (1989), Prentice-Hall · Zbl 0714.93056
[25] Steinwart, I.; Hush, D.; Scovel, C., An explicit description of the reproducing kernel Hilbert space of Gaussian RBF kernels, IEEE Transactions on Information Theory, 52, 4635-4643, (2006) · Zbl 1320.68148
[26] Tikhonov, A. N.; Arsenin, V. Y., Solutions of ill-posed problems, (1977), Winston/Wiley · Zbl 0354.65028
[27] Wahba, G., Spline models for observational data, (1990), SIAM · Zbl 0813.62001
[28] Zeidler, E., Applied functional analysis, (1995), Springer
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.