## Spectrum and multiplicities for restrictions of unitary representations in nilpotent Lie groups.(English)Zbl 0628.22007

Let K be a Lie subgroup of the connected, simply connected nilpotent Lie group G, and let $$\pi$$ $$\in \hat G$$. The main result of the paper is a theorem describing the direct integral decomposition of $$\pi$$ /K into irreducibles.
Here is a brief account of the statement. Let $${\mathfrak g,k}$$ be the corresponding Lie algebras, let $${\mathfrak g^ *,k^ *}$$ be their duals, let $$P:= {\mathfrak g^ *}\to {\mathfrak k}^ *$$ be the canonical projection, and let $${\mathfrak o}_{\pi}\subset {\mathfrak g}^ *$$ be the Kirillov orbit corresponding to $$\pi$$. Almost all Kirillov orbits $${\mathfrak o}_{\sigma}\subset {\mathfrak k*}$$ meeting P($${\mathfrak o}_{\pi})$$ have the same dimension. The direct integral is over the corresponding representations $$\sigma$$ $$\in K{\hat{\;}}$$, and the multiplicity $$m_{\sigma}$$ of $$\sigma$$ is the number of $$Ad^ *(K)$$-orbits in $$P^{-1}({\mathfrak o}_{\sigma})\cap {\mathfrak o}_{\pi}$$. The measure also has a simple description. The numbers $$m_{\sigma}$$ are (almost everywhere) finite or all infinite; they are a.e. finite iff $$\dim {\mathfrak o}_{\pi}+\dim {\mathfrak o}_{\sigma}=2 \dim Ad^ *(K)\ell$$ for generic $$\ell \in {\mathfrak o}_{\sigma}$$. The paper also contains some examples, including a calculation of the decomposition of a tensor product.

### MSC:

 22E47 Representations of Lie and real algebraic groups: algebraic methods (Verma modules, etc.) 22D10 Unitary representations of locally compact groups
Full Text: