##
**The endoscopic classification of representations. Orthogonal and symplectic groups.**
*(English)*
Zbl 1310.22014

Colloquium Publications. American Mathematical Society 61. Providence, RI: American Mathematical Society (AMS) (ISBN 978-0-8218-4990-3/hbk). xviii, 590 p. (2013).

Automorphic representations present one of the most important objects in the representation theory, which has been extensively studied for many years. Automorphic representations carry fundamental arithmetic data and much is known about such representations of the general linear groups.

The goal of this book is to provide a classification of automorphic representations of special orthogonal and symplectic groups in terms of those of the general linear group. Also, the author introduces some new methods and techniques which should have important applications in the classification of automorphic representations of more general reductive groups. In the proofs of the main results, a key role is played by the stable trace formula, which now holds unconditionally due to the recent proof of the fundamental lemma. However, some results of this book are still conditional on the stabilization of twisted trace formulas for the general linear and special odd-orthogonal groups.

We note that the local Langlands correspondence, the Jacquet-Shalika strong multiplicity one and the Mœglin-Waldspurger characterization of the automorphic discrete spectrum in terms of cuspidal automorphic representations are all known in the general linear group case. The author’s intention is to generalize these three theorems to the special orthogonal and symplectic group case. Representations of groups other than the general linear one naturally separate into \(L\)-packets and representations with the same \(L\)-functions and the \(\epsilon\)-factors belong to the same \(L\)-packet. One of the first steps of the author is to introduce objects which will be used in place of the global Langlands group \(L_{F}\) (for a fixed number field \(F\)) and to formulate the theory of endoscopy, together with the appropriate endoscopic datum, in terms of the introduced objects. The global Langlands group \(L_{F}\) is a hypothetical global analogue of the explicit local Langlands group, and it is thought to be a locally compact extension \[ 1 \rightarrow K_F \rightarrow L_F \rightarrow W_F \rightarrow 1 \] of the Weil group \(W_F\) by a compact connected group \(K_F\). To keep things simple, we will describe the main results of the book under review in terms of \(L_F\).

In what follows, \(G\) will denote the quasisplit special orthogonal or symplectic group and \(G(N)\) such group of rank \(N\). We will now describe the main results, which are stated in the first chapter and their proofs are given in the rest of the book.

We will first describe the main local result, which gives a local Langlands parametrization of irreducible representations as a disjoint union of finite \(L\)-packets.

Let \(\Pi(G)\) stand for the set of irreducible representations of \(G(F)\) and let \(\Pi_{u}(G)\) be the subset of unitary representations and \(\Pi_{\mathrm{temp}}(G)\) the subset of tempered representations. We denote by \(\widetilde{\Pi}(G)\) the set of \(\widetilde{Out}_{N}(G)\) orbits in \(\Pi(G)\), where \(\widetilde{Out}_{N}(G)\) is the image of the group of outer automorphisms of \(G\).

Let \(\Psi(N)\) denote the set of formal unordered direct sums \(\psi = l_1 \psi_1 \oplus \cdots \oplus l_r \psi_r\) for positive integers \(l_i\) and distinct elements \(\psi_i = \mu_i \otimes \nu_i\), where \(\mu_i\) is a cuspidal unitary automorphic representation of \(\mathrm{GL}(m_i)\), \(\nu_i\) is the unique irreducible representation of \(\mathrm{SU}(2)\) of degree \(n_i\), and \(N = l_1 m_1 n_1 + \cdots + l_r m_r n_r\).

The subset of self-dual parameters in \(\Psi(N)\) will be denoted by \(\widetilde{\Psi} (N)\). Let \(\widetilde{\Psi} (G) = \{ \psi \in \widetilde{\Psi} (N) : \widetilde{\psi}\) factors through \(^{L}G \}\), where \(\widetilde{\psi}\) stands for the contragredient of \(\psi\). Also, let \(\widetilde{\Psi}_{\mathrm{sim}} (N)\) denote the subset of parameters in \(\widetilde{\Psi} (N)\) which are simple, in the sense that \(r = 1\) and \(l_1 = 1\). Furthermore, let \(\widetilde{\Phi}_{\mathrm{sim}}(N) = \widetilde{\Phi}_{\mathrm{sim}}(G(N))\) be the subset of parameters in \(\widetilde{\Psi}_{\mathrm{sim}} (N)\) which are generic, i.e., such that \(\nu\) is trivial.

For \(\psi \in \widetilde{\Psi} (G)\), \(\mathcal{S}_{\psi}\) will denote the centralizer of the image of \(\widetilde{\psi}_{G}\) in the complex dual group \(\widehat{G}\) of \(G\), where \(\widetilde{\psi}_{G}\) is an \(L\)-embedding of \(\mathcal{L}_{\psi} \times SL(2, \mathbb{C})\) in \(^{L}G\) such that \(\xi \circ \widetilde{\psi}_{G} = \widetilde{\psi}\), for the \(L\)-embedding \(\xi\) from the twisted endoscopic datum. \(\widetilde{\Psi}_2(G)\) will stand for the subset of parameters \(\psi \in \widetilde{\Psi} (G)\) such that the centralizer \(\mathcal{S}_{\psi}\) is finite.

Then we have the following result:

If \(F\) is a local field, then for any local parameter \(\psi \in \widetilde{\Psi} (G)\) there exists a finite set \(\widetilde{\Pi}_{\psi}\) over \(\widetilde{\Pi}_{u}(G)\), equipped with a canonical mapping \(\pi \rightarrow \langle \cdot, \pi \rangle\) from \(\widetilde{\Pi}_{\psi}\) into the group of characters of \(\mathcal{S}_{\psi}\) such that \(\langle \cdot, \pi \rangle = 1\) if \(G\) and \(\pi\) are unramified. If \(\psi \in \widetilde{\Psi}(G)\) is trivial on \(SU(2)\), then the elements in \(\widetilde{\Pi}_{\psi}\) are tempered and multiplicity free, and the introduced mapping is injective. Also, every element in \(\widetilde{\Pi}_{\mathrm{temp}}(G)\) belongs to exactly one packet \(\widetilde{\Pi}_{\psi}\). If \(F\) is nonarchimedean, the introduced mapping is bijective.

We will now discuss the main global results.

We fix the maximal compact subgroup \(K_F\) of \(G(F)\) and write \(\mathcal{H}(G)\) for the corresponding Hecke algebra of smooth left and right \(K_F\)-finite functions of compact support on \(G\). Furthermore, we denote by \(\widetilde{\mathcal{H}}(G)\) the subalgebra of \(\widetilde{Out}_{N}(G)\)-invariant functions in \(\mathcal{H}(G)\).

Then we have the following results:

If \(F\) is a global field, then there is an \(\widetilde{\mathcal{H}}(G)\)-module isomorphism \[ L^{2}_{\mathrm{disc}}(G(F) \setminus G(\mathbb{A})) \cong \bigoplus_{\psi \in \widetilde{\Psi}_2(G)} m_{\psi} \bigg ( \bigoplus_{\pi \in \widetilde{\Pi}_{\psi}(\epsilon_{\psi})} \pi \bigg ), \] where \(m_{\psi}\) equals 1 or 2, while \(\epsilon_{\psi} : \mathcal{S}_{\psi} \rightarrow \{ \pm 1 \}\) is a linear character defined explicitly in terms of symplectic \(\epsilon\)-factors, and \(\widetilde{\Pi}_{\psi}(\epsilon_{\psi})\) is the subset of representations in the global packet \(\widetilde{\Pi}_{\psi}\) such that the character \(\langle \cdot, \pi \rangle\) on \(\mathcal{S}_{\psi}\) equals \(\epsilon_{\psi}\).

Suppose that \(\phi\) belongs to \(\widetilde{\Phi}_{\mathrm{sim}}(N)\). Then \(\widehat{G}\) is orthogonal if and only if the symmetric square \(L\)-function \(L(s, \phi, S^{2})\) has a pole at \(s=1\), while \(\widehat{G}\) is symplectic if and only if the skew-symmetric square \(L\)-function \(L(s, \phi, \Lambda^{2})\) has a pole at \(s=1\). Suppose that, for \(i = 1, 2\), \(\phi_i\) belongs to \(\widetilde{\Phi}_{\mathrm{sim}}(G_i)\), for simple endoscopic data \(G_i = G_i(N)\), which we identify with the quasisplit group. Then the corresponding Rankin-Selberg \(\epsilon\)-factor satisfies \(\epsilon \big (\frac{1}{2}, \phi_1 \times \phi_2 \big ) = 1\) if \(\widehat{G}_1\) and \(\widehat{G}_2\) are either both orthogonal or both symplectic.

In what follows, we describe the content of each chapter, starting with the second chapter (the first one, as has been observed already, contains the basic notation and statements of the main results).

Chapter 2 is devoted to local endoscopy. In this chapter, the author provides a more precise formulation of the main local result an gives a construction of the local packets. Also, this chapter contains some fundamental technical local results.

Chapter 3 is devoted to global endoscopy. In this chapter, the author recalls the discrete part of the trace formula and establishes certain criteria for the vanishing of coefficients, what is used many times later in the book.

In Chapter 4 the contribution of a parameter to the discrete part of the trace formula and to the corresponding endoscopic decomposition are compared. This chapter presents a standard model and explains how the terms appearing in the spectral extension can be matched with those appearing in the endoscopic one.

Chapter 5 is the heart of this book, it provides a connection between local and global results obtained in other chapters. Also, it presents a transition from the general comparisons from previous chapters to the investigation of the exceptional cases which are needed to complete the inductive argument over \(N\).

Chapter 6 deals with generic local parameters and contains the proof of the local Langlands classification for the studied groups (modified by the outer automorphism in the case of \(\mathrm{SO}(2n)\)). In this chapter, proofs are given of the main local results for certain Lanaglands parameters appearing in \(\widetilde{\Phi}_{2}(G)\).

Chapter 7 deals with nongeneric local parameters and contains the proof of the main local results in general. The proofs obtained in this chapter use global parameters, local results for generic parameters obtained in the previous chapters and the Aubert-Schneider-Stuhler duality.

The proofs of the main global theorems are completed in Chapter 8, which presents the end of the long running induction argument, enhanced by local results and refinements of the theorems obtained in previous chapters.

A discussion of the inner forms of orthogonal and symplectic groups is done in Chapter 9. In this case, analogues of the main theorems are given, and the proofs will appear elsewhere.

The goal of this book is to provide a classification of automorphic representations of special orthogonal and symplectic groups in terms of those of the general linear group. Also, the author introduces some new methods and techniques which should have important applications in the classification of automorphic representations of more general reductive groups. In the proofs of the main results, a key role is played by the stable trace formula, which now holds unconditionally due to the recent proof of the fundamental lemma. However, some results of this book are still conditional on the stabilization of twisted trace formulas for the general linear and special odd-orthogonal groups.

We note that the local Langlands correspondence, the Jacquet-Shalika strong multiplicity one and the Mœglin-Waldspurger characterization of the automorphic discrete spectrum in terms of cuspidal automorphic representations are all known in the general linear group case. The author’s intention is to generalize these three theorems to the special orthogonal and symplectic group case. Representations of groups other than the general linear one naturally separate into \(L\)-packets and representations with the same \(L\)-functions and the \(\epsilon\)-factors belong to the same \(L\)-packet. One of the first steps of the author is to introduce objects which will be used in place of the global Langlands group \(L_{F}\) (for a fixed number field \(F\)) and to formulate the theory of endoscopy, together with the appropriate endoscopic datum, in terms of the introduced objects. The global Langlands group \(L_{F}\) is a hypothetical global analogue of the explicit local Langlands group, and it is thought to be a locally compact extension \[ 1 \rightarrow K_F \rightarrow L_F \rightarrow W_F \rightarrow 1 \] of the Weil group \(W_F\) by a compact connected group \(K_F\). To keep things simple, we will describe the main results of the book under review in terms of \(L_F\).

In what follows, \(G\) will denote the quasisplit special orthogonal or symplectic group and \(G(N)\) such group of rank \(N\). We will now describe the main results, which are stated in the first chapter and their proofs are given in the rest of the book.

We will first describe the main local result, which gives a local Langlands parametrization of irreducible representations as a disjoint union of finite \(L\)-packets.

Let \(\Pi(G)\) stand for the set of irreducible representations of \(G(F)\) and let \(\Pi_{u}(G)\) be the subset of unitary representations and \(\Pi_{\mathrm{temp}}(G)\) the subset of tempered representations. We denote by \(\widetilde{\Pi}(G)\) the set of \(\widetilde{Out}_{N}(G)\) orbits in \(\Pi(G)\), where \(\widetilde{Out}_{N}(G)\) is the image of the group of outer automorphisms of \(G\).

Let \(\Psi(N)\) denote the set of formal unordered direct sums \(\psi = l_1 \psi_1 \oplus \cdots \oplus l_r \psi_r\) for positive integers \(l_i\) and distinct elements \(\psi_i = \mu_i \otimes \nu_i\), where \(\mu_i\) is a cuspidal unitary automorphic representation of \(\mathrm{GL}(m_i)\), \(\nu_i\) is the unique irreducible representation of \(\mathrm{SU}(2)\) of degree \(n_i\), and \(N = l_1 m_1 n_1 + \cdots + l_r m_r n_r\).

The subset of self-dual parameters in \(\Psi(N)\) will be denoted by \(\widetilde{\Psi} (N)\). Let \(\widetilde{\Psi} (G) = \{ \psi \in \widetilde{\Psi} (N) : \widetilde{\psi}\) factors through \(^{L}G \}\), where \(\widetilde{\psi}\) stands for the contragredient of \(\psi\). Also, let \(\widetilde{\Psi}_{\mathrm{sim}} (N)\) denote the subset of parameters in \(\widetilde{\Psi} (N)\) which are simple, in the sense that \(r = 1\) and \(l_1 = 1\). Furthermore, let \(\widetilde{\Phi}_{\mathrm{sim}}(N) = \widetilde{\Phi}_{\mathrm{sim}}(G(N))\) be the subset of parameters in \(\widetilde{\Psi}_{\mathrm{sim}} (N)\) which are generic, i.e., such that \(\nu\) is trivial.

For \(\psi \in \widetilde{\Psi} (G)\), \(\mathcal{S}_{\psi}\) will denote the centralizer of the image of \(\widetilde{\psi}_{G}\) in the complex dual group \(\widehat{G}\) of \(G\), where \(\widetilde{\psi}_{G}\) is an \(L\)-embedding of \(\mathcal{L}_{\psi} \times SL(2, \mathbb{C})\) in \(^{L}G\) such that \(\xi \circ \widetilde{\psi}_{G} = \widetilde{\psi}\), for the \(L\)-embedding \(\xi\) from the twisted endoscopic datum. \(\widetilde{\Psi}_2(G)\) will stand for the subset of parameters \(\psi \in \widetilde{\Psi} (G)\) such that the centralizer \(\mathcal{S}_{\psi}\) is finite.

Then we have the following result:

If \(F\) is a local field, then for any local parameter \(\psi \in \widetilde{\Psi} (G)\) there exists a finite set \(\widetilde{\Pi}_{\psi}\) over \(\widetilde{\Pi}_{u}(G)\), equipped with a canonical mapping \(\pi \rightarrow \langle \cdot, \pi \rangle\) from \(\widetilde{\Pi}_{\psi}\) into the group of characters of \(\mathcal{S}_{\psi}\) such that \(\langle \cdot, \pi \rangle = 1\) if \(G\) and \(\pi\) are unramified. If \(\psi \in \widetilde{\Psi}(G)\) is trivial on \(SU(2)\), then the elements in \(\widetilde{\Pi}_{\psi}\) are tempered and multiplicity free, and the introduced mapping is injective. Also, every element in \(\widetilde{\Pi}_{\mathrm{temp}}(G)\) belongs to exactly one packet \(\widetilde{\Pi}_{\psi}\). If \(F\) is nonarchimedean, the introduced mapping is bijective.

We will now discuss the main global results.

We fix the maximal compact subgroup \(K_F\) of \(G(F)\) and write \(\mathcal{H}(G)\) for the corresponding Hecke algebra of smooth left and right \(K_F\)-finite functions of compact support on \(G\). Furthermore, we denote by \(\widetilde{\mathcal{H}}(G)\) the subalgebra of \(\widetilde{Out}_{N}(G)\)-invariant functions in \(\mathcal{H}(G)\).

Then we have the following results:

If \(F\) is a global field, then there is an \(\widetilde{\mathcal{H}}(G)\)-module isomorphism \[ L^{2}_{\mathrm{disc}}(G(F) \setminus G(\mathbb{A})) \cong \bigoplus_{\psi \in \widetilde{\Psi}_2(G)} m_{\psi} \bigg ( \bigoplus_{\pi \in \widetilde{\Pi}_{\psi}(\epsilon_{\psi})} \pi \bigg ), \] where \(m_{\psi}\) equals 1 or 2, while \(\epsilon_{\psi} : \mathcal{S}_{\psi} \rightarrow \{ \pm 1 \}\) is a linear character defined explicitly in terms of symplectic \(\epsilon\)-factors, and \(\widetilde{\Pi}_{\psi}(\epsilon_{\psi})\) is the subset of representations in the global packet \(\widetilde{\Pi}_{\psi}\) such that the character \(\langle \cdot, \pi \rangle\) on \(\mathcal{S}_{\psi}\) equals \(\epsilon_{\psi}\).

Suppose that \(\phi\) belongs to \(\widetilde{\Phi}_{\mathrm{sim}}(N)\). Then \(\widehat{G}\) is orthogonal if and only if the symmetric square \(L\)-function \(L(s, \phi, S^{2})\) has a pole at \(s=1\), while \(\widehat{G}\) is symplectic if and only if the skew-symmetric square \(L\)-function \(L(s, \phi, \Lambda^{2})\) has a pole at \(s=1\). Suppose that, for \(i = 1, 2\), \(\phi_i\) belongs to \(\widetilde{\Phi}_{\mathrm{sim}}(G_i)\), for simple endoscopic data \(G_i = G_i(N)\), which we identify with the quasisplit group. Then the corresponding Rankin-Selberg \(\epsilon\)-factor satisfies \(\epsilon \big (\frac{1}{2}, \phi_1 \times \phi_2 \big ) = 1\) if \(\widehat{G}_1\) and \(\widehat{G}_2\) are either both orthogonal or both symplectic.

In what follows, we describe the content of each chapter, starting with the second chapter (the first one, as has been observed already, contains the basic notation and statements of the main results).

Chapter 2 is devoted to local endoscopy. In this chapter, the author provides a more precise formulation of the main local result an gives a construction of the local packets. Also, this chapter contains some fundamental technical local results.

Chapter 3 is devoted to global endoscopy. In this chapter, the author recalls the discrete part of the trace formula and establishes certain criteria for the vanishing of coefficients, what is used many times later in the book.

In Chapter 4 the contribution of a parameter to the discrete part of the trace formula and to the corresponding endoscopic decomposition are compared. This chapter presents a standard model and explains how the terms appearing in the spectral extension can be matched with those appearing in the endoscopic one.

Chapter 5 is the heart of this book, it provides a connection between local and global results obtained in other chapters. Also, it presents a transition from the general comparisons from previous chapters to the investigation of the exceptional cases which are needed to complete the inductive argument over \(N\).

Chapter 6 deals with generic local parameters and contains the proof of the local Langlands classification for the studied groups (modified by the outer automorphism in the case of \(\mathrm{SO}(2n)\)). In this chapter, proofs are given of the main local results for certain Lanaglands parameters appearing in \(\widetilde{\Phi}_{2}(G)\).

Chapter 7 deals with nongeneric local parameters and contains the proof of the main local results in general. The proofs obtained in this chapter use global parameters, local results for generic parameters obtained in the previous chapters and the Aubert-Schneider-Stuhler duality.

The proofs of the main global theorems are completed in Chapter 8, which presents the end of the long running induction argument, enhanced by local results and refinements of the theorems obtained in previous chapters.

A discussion of the inner forms of orthogonal and symplectic groups is done in Chapter 9. In this case, analogues of the main theorems are given, and the proofs will appear elsewhere.

Reviewer: Ivan Matić (Osijek)

### MSC:

22E50 | Representations of Lie and linear algebraic groups over local fields |

22E55 | Representations of Lie and linear algebraic groups over global fields and adèle rings |

11F66 | Langlands \(L\)-functions; one variable Dirichlet series and functional equations |

11F37 | Forms of half-integer weight; nonholomorphic modular forms |

11F70 | Representation-theoretic methods; automorphic representations over local and global fields |

58C40 | Spectral theory; eigenvalue problems on manifolds |