##
**Matrix-valued kernels for shape deformation analysis.**
*(English)*
Zbl 1396.46025

Summary: The main purpose of this paper is providing a systematic study and classification of non-scalar positive definite kernels for Reproducing Kernel Hilbert Spaces (RKHS), to be used in the analysis of deformations in shape spaces endowed with metrics induced by the action of groups of diffeomorphisms. After providing an introduction to matrix-valued kernels and their relevant differential properties, we explore extensively those, that we call TRI kernels, that induce a metric on the corresponding Hilbert spaces of vector fields that is both translation- and rotation-invariant. These are analyzed in a very effective manner in the Fourier domain, where the characterization of RKHS of curl-free and divergence-free vector fields is particularly natural. A technique for constructing generic matrix-valued kernels from scalar kernels is also developed. The exposition of the theory is supported by several examples. We also provide numerical results that show the dynamics induced by different choices of TRI kernels on the manifold of labeled landmark points, and illustrate one application in computational anatomy.

### MSC:

46E22 | Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) |

58D05 | Groups of diffeomorphisms and homeomorphisms as manifolds |

94A08 | Image processing (compression, reconstruction, etc.) in information and communication theory |