Vershik, A. M.; Graev, M. I. Structure of the complementary series and special representations of the groups \(O(n,1)\) and \(U(n,1)\). (English. Russian original) Zbl 1148.22017 Russ. Math. Surv. 61, No. 5, 799-884 (2006); translation from Usp. Mat. Nauk 61, No. 5, 3-88 (2006). A special unitary representation \((\pi,\mathcal H)\) of a Lie group \(G\) is one, for which there exists a non-trivial cocycle \(b\colon G\to \mathcal H\) which can be used to define a representation \((\hat\pi,\mathcal H\oplus \mathbb C)\) via \(\hat \pi(g)(v,z)=(\pi(g)v+zb(g),z)\). The interest in special representations initially comes from the fact that they cannot be separated from the identity representation in the Fell topology. Thus they can occur only if \(G\) does not have Kazhdan’s property \((T)\). The authors have shown that special representations can be used to construct representations of current groups with values in \(G\). In this paper, the authors give explicit descriptions of the special representations of the groups \(O(n, 1)\) and \(U(n, 1)\). In fact, they do much more: They present a number of different realizations of the complementary series representations of these groups and show how the special representations occur in the limiting cases. Special emphasis is put on the fact that here the restrictions of special representations to a maximal parabolic subgroup remain irreducible. Reviewer: Joachim Hilgert (Paderborn) Cited in 11 Documents MSC: 22E65 Infinite-dimensional Lie groups and their Lie algebras: general properties 22D10 Unitary representations of locally compact groups 20G20 Linear algebraic groups over the reals, the complexes, the quaternions Keywords:special representation; complementary series; property (T) PDFBibTeX XMLCite \textit{A. M. Vershik} and \textit{M. I. Graev}, Russ. Math. Surv. 61, No. 5, 799--884 (2006; Zbl 1148.22017); translation from Usp. Mat. Nauk 61, No. 5, 3--88 (2006) Full Text: DOI