##
**Rigidity, unitary representations of semisimple groups, and fundamental groups of manifolds with rank one transformation group.**
*(English)*
Zbl 0970.22011

Let \(G\) be a simple Lie group locally isomorphic to \(SO(n,1)\) or \(SU(n,1),\) and let \(\Lambda\) be a discrete subgroup of \(G.\) The critical exponent \(\delta(\Lambda)\) of \(\Lambda\) is
\[
\delta(\Lambda)=\inf \{s \mid \sum_{\lambda\in \Lambda}e^{-sd(\lambda x_0,x_0)}<\infty\},
\]
where \(d\) is a \(G\)-invariant metric on the associated Riemannian symmetric space \(G/K\) and \(x_0\) is a fixed origin in \(G/K.\) One of the main results of this truly remarkable paper is as follows.

Theorem A. Assume that \(G\) is not locally isomorphic to \(SL(2, {\mathbb R}),\) and let \(\Gamma\) be a lattice in \(G.\) If \(\Gamma\) is isomorphic to a discrete subgroup \(\Lambda\) of some \(SO(m,1)\) or \(SU(m,1),\) then \(\delta(\Gamma)\leq \delta(\Lambda).\) This generalizes a result of C. Yue [Ann. Math. 143, 331-355 (1996; Zbl 0843.22019)] (see also G. Besson, G. Courtois and S. Gallot [Acta Math. 183, 145-169 (1996; Zbl 1035.53038)] and M. Bourdon [IHES Publ. Math. 183, 95-104 (1996; Zbl 0883.53047)]). The above theorem is extended in order to obtain a rigidity result in the situation where \(G\) acts on a compact manifold \(M,\) preserving a geometric structure, and \(\Lambda\) is the fundamental group of \(M.\) One of the most interesting aspects of this paper is the novelty of the approach. It is based on the study of unitary representations of the Lie group \(G\), more precisely, on estimates of their matrix coefficients and on their cohomology. The key notion is the following invariant \(p(G)\) which makes sense for an arbitrary group: \(p(G)\) is the infimum over all real numbers \(0\leq p\leq \infty\) for which there exists a unitary representation \((\pi, {\mathcal H})\) of \(G\) with non-vanishing first cohomology group \(H^1(G,\pi)\) which is strongly \(L^p\)( that is, the matrix coefficients of \(\pi\) corresponding to a dense subspace of \({\mathcal H}\) are in \(L^{p+\varepsilon}(G)\) for all \(\varepsilon>0\)). A crucial result in the paper is the following:

Theorem B. With notation and hypotheses as in Theorem A, one has: \(p(G)=p(\Gamma)=\delta(\Gamma)=\delta (G)\) (where \(\delta (G)=n-1\) if \(G=SO(n,1)\) and \(\delta(G)=2n\) if \(G=SU(n,1)\)). The proof is based in an essential way on the following remarkable result.

Theorem C. With notation and hypotheses as in Theorem A, assume moreover that \(G\) is not locally isomorphic to \(SL(2, {\mathbb C}).\) Let \(\pi\) be a unitary representation of \(\Gamma.\) Then \(H^1(\Gamma,\pi)\) is isomorphic to \(H^1(G,\text{Ind}\pi),\) where \(\text{Ind}\pi\) is the unitary representation of \(G\) induced by \(\pi.\) This result is known (and easy to prove) in case \(\Gamma\) is cocompact, so that the real (and difficult) issue is when \(\Gamma\) is not cocompact. In fact, this result may fail for (non-cocompact) lattices in the groups excluded in the above theorem, that is, \(SL(2,{\mathbb R}) \) or \(SL(2,{\mathbb C}).\)

Theorem A. Assume that \(G\) is not locally isomorphic to \(SL(2, {\mathbb R}),\) and let \(\Gamma\) be a lattice in \(G.\) If \(\Gamma\) is isomorphic to a discrete subgroup \(\Lambda\) of some \(SO(m,1)\) or \(SU(m,1),\) then \(\delta(\Gamma)\leq \delta(\Lambda).\) This generalizes a result of C. Yue [Ann. Math. 143, 331-355 (1996; Zbl 0843.22019)] (see also G. Besson, G. Courtois and S. Gallot [Acta Math. 183, 145-169 (1996; Zbl 1035.53038)] and M. Bourdon [IHES Publ. Math. 183, 95-104 (1996; Zbl 0883.53047)]). The above theorem is extended in order to obtain a rigidity result in the situation where \(G\) acts on a compact manifold \(M,\) preserving a geometric structure, and \(\Lambda\) is the fundamental group of \(M.\) One of the most interesting aspects of this paper is the novelty of the approach. It is based on the study of unitary representations of the Lie group \(G\), more precisely, on estimates of their matrix coefficients and on their cohomology. The key notion is the following invariant \(p(G)\) which makes sense for an arbitrary group: \(p(G)\) is the infimum over all real numbers \(0\leq p\leq \infty\) for which there exists a unitary representation \((\pi, {\mathcal H})\) of \(G\) with non-vanishing first cohomology group \(H^1(G,\pi)\) which is strongly \(L^p\)( that is, the matrix coefficients of \(\pi\) corresponding to a dense subspace of \({\mathcal H}\) are in \(L^{p+\varepsilon}(G)\) for all \(\varepsilon>0\)). A crucial result in the paper is the following:

Theorem B. With notation and hypotheses as in Theorem A, one has: \(p(G)=p(\Gamma)=\delta(\Gamma)=\delta (G)\) (where \(\delta (G)=n-1\) if \(G=SO(n,1)\) and \(\delta(G)=2n\) if \(G=SU(n,1)\)). The proof is based in an essential way on the following remarkable result.

Theorem C. With notation and hypotheses as in Theorem A, assume moreover that \(G\) is not locally isomorphic to \(SL(2, {\mathbb C}).\) Let \(\pi\) be a unitary representation of \(\Gamma.\) Then \(H^1(\Gamma,\pi)\) is isomorphic to \(H^1(G,\text{Ind}\pi),\) where \(\text{Ind}\pi\) is the unitary representation of \(G\) induced by \(\pi.\) This result is known (and easy to prove) in case \(\Gamma\) is cocompact, so that the real (and difficult) issue is when \(\Gamma\) is not cocompact. In fact, this result may fail for (non-cocompact) lattices in the groups excluded in the above theorem, that is, \(SL(2,{\mathbb R}) \) or \(SL(2,{\mathbb C}).\)

Reviewer: Mohamed B.Bekka (Metz)