The Penrose transform for \(\text{Sp}(n,\mathbf R)\) and singular unitary representations. (English) Zbl 1020.32015

Sekiguchi, Hideko

J. Math. Soc. Japan 54, No. 1, 215-253 (2002).

Summary: We give a general definition of the Radon-Penrose transform for a Zuckerman-Vogan derived functor module of a reductive Lie group \(G\), which maps from the Dolbeault cohomology group over a pseudo-Kähler homogeneous manifold into the space of smooth sections of a vector bundle over a Riemannian symmetric space. Furthermore, we formulate a functorial property between two Penrose transforms in the context of the Kobayashi theory of discretely decomposable restrictions of unitary representations.
Based on this general theory, we study the Penrose transform for a family of singular unitary representations of \(\text{Sp}(n,\mathbb{R})\) in details. We prove that the image of the Penrose transform is exactly the space of global holomorphic solutions of the system of partial differential equations of minor determinant type of odd degree over the bounded symmetric domain of type CI, which is biholomorphic to the Siegel upper half space. This system might be regarded as a generalization of the Gauss-Aomoto-Gelfand hypergeometric differential equations to higher order. We also find a new phenomenon that the kernel of the Penrose transform is non-zero, which we determine explicitly by means of representation theory.

MSC:

32L25	Twistor theory, double fibrations (complex-analytic aspects)
22E30	Analysis on real and complex Lie groups

Keywords:

Radon-Penrose transform; singular unitary representations of \(\text{Sp}(n,\mathbb{R})\)