The unitary dual for complex classical Lie groups.

*(English)*Zbl 0692.22006The paper under review gives a complete classification of the unitary dual for the complex classical Lie groups.

Recall that there are two well known ways of constructing unitary representations from lower groups: unitary induction from a parabolic subgroup and complementary series. A reasonable way to describe the unitary dual is to find a distinguished set of representations for each group such that every other unitary representation is obtained from this set by the above constructions from strictly smaller parabolic subgroups.

For integral infinitesimal character such a set is provided by the special unipotent representations introduced by D. Barbasch and D. Vogan [Ann. Math., II. Ser. 121, 41-110 (1985; Zbl 0582.22007)]. They were shown to have many interesting properties such as the fact that their annihilator is a maximal primitive ideal. From this character formulas were obtained for them.

It turns out that in general this set is not large enough to describe the entire unitary dual. The definition of special unipotent representations suggests that one should consider representations with maximal primitive ideal and infinitesimal character as small as possible. However it is not clear which of them should be unitary. This leads to the task of explicitly computing the \({\mathfrak g}\)-invariant form and ruling out the cases when it is indefinite. For the classical groups various techniques introduced here simplify the calculations significantly. A first step reduces the question of computing the signature on isolated representations (the most difficult case) to the similar question for a simpler representation but on a higher rank group. Using the “bottom layer K-type” idea by B. Speh and D. Vogan [Acta Math. 145, 227-299 (1980; Zbl 0457.22011)] one comes down to the question of computing the signature of simpler representations on a higher rank group. In particular the finite set consists of K-types which are very close to the ones with fundamental extremal weight. These computations are then performed to obtain the complete classification. Some low rank examples are given.

The case \(GL_ n\) was previously dealt with by D. Vogan [Invent. Math. 83, 449-505 (1986; Zbl 0598.22008)]. The case of regular integral infinitesimal character was done by T. Enright [Duke Math. J. 46, 513-525 (1979; Zbl 0427.22010)].

Recall that there are two well known ways of constructing unitary representations from lower groups: unitary induction from a parabolic subgroup and complementary series. A reasonable way to describe the unitary dual is to find a distinguished set of representations for each group such that every other unitary representation is obtained from this set by the above constructions from strictly smaller parabolic subgroups.

For integral infinitesimal character such a set is provided by the special unipotent representations introduced by D. Barbasch and D. Vogan [Ann. Math., II. Ser. 121, 41-110 (1985; Zbl 0582.22007)]. They were shown to have many interesting properties such as the fact that their annihilator is a maximal primitive ideal. From this character formulas were obtained for them.

It turns out that in general this set is not large enough to describe the entire unitary dual. The definition of special unipotent representations suggests that one should consider representations with maximal primitive ideal and infinitesimal character as small as possible. However it is not clear which of them should be unitary. This leads to the task of explicitly computing the \({\mathfrak g}\)-invariant form and ruling out the cases when it is indefinite. For the classical groups various techniques introduced here simplify the calculations significantly. A first step reduces the question of computing the signature on isolated representations (the most difficult case) to the similar question for a simpler representation but on a higher rank group. Using the “bottom layer K-type” idea by B. Speh and D. Vogan [Acta Math. 145, 227-299 (1980; Zbl 0457.22011)] one comes down to the question of computing the signature of simpler representations on a higher rank group. In particular the finite set consists of K-types which are very close to the ones with fundamental extremal weight. These computations are then performed to obtain the complete classification. Some low rank examples are given.

The case \(GL_ n\) was previously dealt with by D. Vogan [Invent. Math. 83, 449-505 (1986; Zbl 0598.22008)]. The case of regular integral infinitesimal character was done by T. Enright [Duke Math. J. 46, 513-525 (1979; Zbl 0427.22010)].

Reviewer: J.Schwermer

##### MSC:

22E46 | Semisimple Lie groups and their representations |

22E30 | Analysis on real and complex Lie groups |

22E10 | General properties and structure of complex Lie groups |

##### Keywords:

unitary dual; complex classical Lie groups; unitary representations; unitary induction; complementary series; parabolic subgroups; integral infinitesimal character; special unipotent representations; maximal primitive ideal; character formulas; fundamental extremal weight##### References:

[1] | Arthur, J.: On some problems suggested by the trace formula. (Lect. Notes in Math., Vol. 1041, pp. 1-50) Berlin-Heidelberg-New York: Springer 1984 · Zbl 0541.22011 |

[2] | [BV1] Barbasch, D., Vogan, D.: Unipotent representations of complex semisimple Lie groups. Ann. Math.121, 41-110 (1985) · Zbl 0582.22007 · doi:10.2307/1971193 |

[3] | [BV2] Barbasch, D., Vogan, D.: Primitive ideals and orbital integrals in complex classical groups. Math. Ann.259, 153-199 (1982) · Zbl 0489.22010 · doi:10.1007/BF01457308 |

[4] | [BV3] Barbasch, D., Vogan, D.: Primitive ideals and orbital integrals in complex exceptional groups. J. Algebra80, 350-382 (1983) · Zbl 0513.22009 · doi:10.1016/0021-8693(83)90006-6 |

[5] | [BG] Bernstein, I.N., Gelfand, S.I.: Tensor products of finite and infinite dimensional representations of semisimple Lie algebras. Compos. Math.41, 245-285 (1980) · Zbl 0445.17006 |

[6] | [BC] Boe, B., Collingwood, D.: A multiplicity one theorem for holomorphically induced modules. Math. Z.192, 265-282 (1986) · Zbl 0598.22009 · doi:10.1007/BF01179428 |

[7] | [Bou] Bourbaki, N.: Groupes et algebres de Lie. Paris Hermann 1968 |

[8] | [D1] Duflo, M.: Representations irreductibles des groupes semisimples complexes. (Lect. Notes in Math., Vol. 497, pp. 26-88). Berlin-Heidelberg-New York: Springer 1975 |

[9] | [D2] Duflo, M.: Representations unitaires irreducibles des groupes semisimples complexes de rang deux. Bull. Soc. Math. France107, 55-96 (1979) · Zbl 0407.22014 |

[10] | [E] Enright, T.: Relative Lie algebra cohomology and unitary representations of complex semisimple Lie groups. Duke Math. J.46, 513-525 (1979) · Zbl 0427.22010 · doi:10.1215/S0012-7094-79-04626-X |

[11] | [ES] Enright, T., Shelton, B.: Categories of highest weight modules: applications to classical hermitian pairs. Mem. Am. Math. Soc. (to appear) · Zbl 0621.17004 |

[12] | [J] Joseph, A.: Dixmier’s problem for Verma and principal series submodules. J. London Math. Soc.20, 193-204 (1979) · Zbl 0421.17005 · doi:10.1112/jlms/s2-20.2.193 |

[13] | [KP] Kraft, H., Procesi, C.: On the geometry of conjugacy classes in classical groups. Comment. Math. Helv.57, 539-601 (1982) · Zbl 0511.14023 · doi:10.1007/BF02565876 |

[14] | [KZ] Knapp, A., Zuckerman, G.: Classification theorems for representations of semisimple Lie groups. (Lect. Notes in Math., Vol. 587, pp. 138-159) Berlin-Heidelberg-New York: Springer 1976 |

[15] | [L] Lusztig, G.: Characters of reductive groups over a finite field. Ann. Math. Stud.107, 1-384 (1984) · Zbl 0556.20033 |

[16] | [SV] Speh, B., Vogan, D.: Reducibility of generalized principal series representations. Acta Math.145, 227-299 (1980) · Zbl 0457.22011 · doi:10.1007/BF02414191 |

[17] | [V1] Vogan, D.: Representations of real reductive Lie groups. Boston: BirkhĂ¤user 1981 · Zbl 0469.22012 |

[18] | [V2] Vogan, D.: Unitarizability of certain series of representations. Ann. Math.120, 141-187 (1984) · Zbl 0561.22010 · doi:10.2307/2007074 |

[19] | [V3] Vogan, D.: The unitary dual ofGL(n). Invent. Math.83, 449-505 (1986) · Zbl 0598.22008 · doi:10.1007/BF01394418 |

[20] | [V4] Vogan, D.: Irreducible characters of semisimple Lie groups III, proof of the Kazhdan-Lusztig conjectures in the integral case. Invent. Math.71, 381-417 (1983) · Zbl 0505.22016 · doi:10.1007/BF01389104 |

[21] | [Z] Zuckerman, G.: Tensor products of finite and infinite dimensional representations of semisimple Lie groups. Ann. Math.106, 295-308 (1977) · Zbl 0384.22004 · doi:10.2307/1971097 |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.