van den Ban, E. P.; Schlichtkrull, H. The most continuous part of the Plancherel decomposition for a reductive symmetric space. (English) Zbl 0878.43018 Ann. Math. (2) 145, No. 2, 267-364 (1997). Let \(G/H\) be a semisimple symmetric space; that is, \(G\) is a connected semisimple real Lie group with involution \(\sigma\), and \(H\) is an open subgroup of the group of fixed points for \(\sigma\) in \(G\). A fundamental problem of harmonic analysis on \(G/H\) is to obtain an explicit direct integral (Plancherel) decomposition of \(L\simeq \int^\oplus_{\widehat G} m_\pi\pi d\mu (\pi)\) of the regular representation \(L\) of \(G\) in \(L^2 (G/H)\) into irreducible unitary representations. The principal result of this paper is the determination of such a decomposition for the part of \(L^2(G/H)\) which corresponds to the “most continuous part” of the spectrum. Let \({\mathfrak g}\) be the Lie algebra of \(G\) and let \({\mathfrak g}= {\mathfrak h}+ {\mathfrak q}\) be its decomposition into \(\pm 1\)-eigenspaces for \(\sigma\) (so that \({\mathfrak h}\) is the Lie algebra of \(H)\). Furthermore, let \(\theta\) be a Cartan involution commuting with \(\sigma\). By analogy with the group case one expects \(L^2(G/H)\) to decompose into a finite number of parts, each attached to a particular parabolic subgroup \(P\) satisfying \(\sigma\theta (P)=P\). For a \(\sigma\)-minimal parabolic subgroup \(P=MAN\) let \(\xi\) be a finite dimensional unitary representation of \(M\) and let \(\lambda\in {\mathfrak a}^*_{q\mathbb{C}}\), where \({\mathfrak a}_q ={\mathfrak a} \cap {\mathfrak q}\). Furthermore, let \(C^\infty (\xi: \lambda)\) and \(C^{-\infty} (\xi: \lambda)\) denote the spaces of smooth, respectively generalized, vectors for the representation \(\pi_{\xi, \lambda}\) associated to \(\sigma\) and \(\lambda\), and let \(C^\infty (\xi: \lambda)^H\) be the space of \(H\)-fixed elements in \(C^{-\infty} (\xi: \lambda)\). In [E. P. van den Ban, Ann. Sci. Éc. Norm. Supér, IV. Sér. 21, 359-412 (1988; Zbl 0714.22009)] and [E. P. van den Ban and H. Schlichtkrull, Fourier transforms on a semisimple symmetric space (to appear in Invent. Math.)] a certain finite dimensional space \(V(\xi)\) (independent of \(\lambda)\) was defined together with a natural family of linear maps \(j^0(\xi: \lambda) =j^0 (P:\xi: \lambda): V(\xi)\to C^{-\infty} (\xi: \lambda)^H\), depending meromorphically on \(\lambda\in {\mathfrak a}^*_{q\mathbb{C}}\) and bijective for generic \(\lambda\). The definition involved a normalization procedure which guarantees that \(\lambda \mapsto j^0 (\xi: \lambda)\) is regular at the imaginary points \(\lambda\in i {\mathfrak a}^*_q\). The Fourier transform \(\widehat f\) for \(f\in C^\infty_c (G/H)\) is then defined by \[ \widehat f(\xi: \lambda)= \int_{G/H} f(gH) \pi_{\xi,- \lambda} (g)j^0 (\xi: \lambda) d(gH)\in \operatorname{Hom} \bigl(V(\xi), C^\infty(\xi: -\lambda) \bigr). \] The main result of the paper asserts that if one sets \(d\mu (\xi,\lambda) =\dim(\xi)d \lambda\), where \(d\lambda\) is the suitably normalized Lebesgue measure on \(i{\mathfrak a}^*_q\), then \(f\mapsto \widehat f\) extends to a partial isometry of \(L^2(G/H)\) onto the space \(\int^\oplus_{\xi, \lambda} m_\xi \pi_{\xi, \lambda} d\mu (\xi,\lambda)\) which specified domains for \(\xi\) and \(\lambda\). The proof of these results uses uniform estimates for the coefficients of a converging expansion for the Eisenstein integral, estimates of Paley-Wiener type for the Fourier transform \({\mathcal F}f\) of a Schwartz function \(f\), the wave packet transform \({\mathcal I} \varphi\) of a \(^0{\mathcal C} (\tau)\)-valued function \(\varphi\) on \(i{\mathfrak a}^*\) on \(G/H\) and other methods. In the last section the authors derive a Paley-Wiener type result for spaces of split rank one. The image by \({\mathcal F}\) of the space of compactly supported smooth functions of type \(\tau\) is characterized by a growth condition. Reviewer: J.Ludwig (Metz) Cited in 1 ReviewCited in 30 Documents MSC: 43A85 Harmonic analysis on homogeneous spaces 22E30 Analysis on real and complex Lie groups 53C35 Differential geometry of symmetric spaces 22E46 Semisimple Lie groups and their representations Keywords:Plancherel decomposition; semisimple symmetric space; semisimple real Lie group; harmonic analysis; regular representation; Lie algebra; Fourier transform; Eisenstein integral Citations:Zbl 0714.22009 PDF BibTeX XML Cite \textit{E. P. van den Ban} and \textit{H. Schlichtkrull}, Ann. Math. (2) 145, No. 2, 267--364 (1997; Zbl 0878.43018) Full Text: DOI Link