##
**Analysis on the minimal representation of \(\mathrm O(p,q)\). II: Branching laws.**
*(English)*
Zbl 1049.22006

[For Part I, see ibid. 180, No. 2, 486–512 (2003; Zbl 1046.22004).]

Let \(G\) be a Lie group and \(G'\) its subgroup, \(\widehat G\) the unitary dual of \(G\). If \(\pi\in\widehat G\), the restriction \(\pi_{\widehat G'}\) is generally not irreducible and we have an irreducible decomposition \[ \pi_{G'}\cong\int_{\widehat G'}m_{\pi}(\tau)\tau \,d\mu(\tau). \] This is the so-called branching law. The present paper is the second in a series of papers devoted to the analysis of the minimal representation \(\widetilde\omega^{p,q}\) of the indefinite orthogonal group \(G = O(p,q)\), i.e., from the point of view of conformal geometry, the authors find the restriction of \(\widetilde\omega^{p,q}\) with respect to the symmetric pair \((G,G') = (O(p,q),O(p'q')\times O(p'',q''))\): A canonical conclusion obtained by them is the branching law for \(O(pq)\downarrow O(p,q')\times O(q'')\): If \(q''\geq 1\) and \(q'+ q'' = q\), the twisted pull-back \(\widetilde\Phi{*_1}\) of the local conformal map between spheres and hyperboloids gives an explicit irreducible decomposition of the unitary representation \(\tilde\omega^{p,q}\) when restricted to \(O(p,q'')\times O(q'')\) \[ \widetilde\Phi*_1: \widetilde\omega^{p,q}|_{O(p,q')\times O(q'')} \to \sum_{l=0}^{\infty} \pi^{p,q'}_{+,l+q''/2-1}\otimes \mathcal H^l (R^{q''}), \] where the \(\mathcal H^l(R^q)\) are the usual spherical harmonics for the compact orthogonal group \(O(q)\) and the \(\pi^{p,q}_{+,\lambda}\) are the representations of the non-compact orthogonal group \(O(p,q)\), which can be regarded as discrete series representations on hyperboloids, or as cohomological induced representations from characters of certain \(\theta\)-stable parabolic subalgebras. The authors give also the Parseval-Plancherel formula for the above restriction: let \(\widetilde\bigtriangleup_M\) be the Yamabe operator on \(M = S^{p-1}\times S^{q-1}\) and \(F\in \operatorname{Ker} \widetilde\bigtriangleup_M\) and develop \(F\) as \(f =\sum_{l=0}^{\infty} F_l^{(1)}F_l^{(2)}\) according to the above decomposition, then \[ \| F\|^2_{\widetilde\omega^{p,q}}= \sum_{l=0}^{\infty}\| F^{(1)}\|^2_{\pi^{p,q'}_{+,l+q''/2-1}} \| F^{(2)}\|^2_{L^2 (S^{q''}-1)}. \] Finally the authors use certain Sobolev estimates to construct infinitely many discrete spectra when both factors in \(G'\) are non-compact, they also conjecture the form of the fall discrete spectrum.

Let \(G\) be a Lie group and \(G'\) its subgroup, \(\widehat G\) the unitary dual of \(G\). If \(\pi\in\widehat G\), the restriction \(\pi_{\widehat G'}\) is generally not irreducible and we have an irreducible decomposition \[ \pi_{G'}\cong\int_{\widehat G'}m_{\pi}(\tau)\tau \,d\mu(\tau). \] This is the so-called branching law. The present paper is the second in a series of papers devoted to the analysis of the minimal representation \(\widetilde\omega^{p,q}\) of the indefinite orthogonal group \(G = O(p,q)\), i.e., from the point of view of conformal geometry, the authors find the restriction of \(\widetilde\omega^{p,q}\) with respect to the symmetric pair \((G,G') = (O(p,q),O(p'q')\times O(p'',q''))\): A canonical conclusion obtained by them is the branching law for \(O(pq)\downarrow O(p,q')\times O(q'')\): If \(q''\geq 1\) and \(q'+ q'' = q\), the twisted pull-back \(\widetilde\Phi{*_1}\) of the local conformal map between spheres and hyperboloids gives an explicit irreducible decomposition of the unitary representation \(\tilde\omega^{p,q}\) when restricted to \(O(p,q'')\times O(q'')\) \[ \widetilde\Phi*_1: \widetilde\omega^{p,q}|_{O(p,q')\times O(q'')} \to \sum_{l=0}^{\infty} \pi^{p,q'}_{+,l+q''/2-1}\otimes \mathcal H^l (R^{q''}), \] where the \(\mathcal H^l(R^q)\) are the usual spherical harmonics for the compact orthogonal group \(O(q)\) and the \(\pi^{p,q}_{+,\lambda}\) are the representations of the non-compact orthogonal group \(O(p,q)\), which can be regarded as discrete series representations on hyperboloids, or as cohomological induced representations from characters of certain \(\theta\)-stable parabolic subalgebras. The authors give also the Parseval-Plancherel formula for the above restriction: let \(\widetilde\bigtriangleup_M\) be the Yamabe operator on \(M = S^{p-1}\times S^{q-1}\) and \(F\in \operatorname{Ker} \widetilde\bigtriangleup_M\) and develop \(F\) as \(f =\sum_{l=0}^{\infty} F_l^{(1)}F_l^{(2)}\) according to the above decomposition, then \[ \| F\|^2_{\widetilde\omega^{p,q}}= \sum_{l=0}^{\infty}\| F^{(1)}\|^2_{\pi^{p,q'}_{+,l+q''/2-1}} \| F^{(2)}\|^2_{L^2 (S^{q''}-1)}. \] Finally the authors use certain Sobolev estimates to construct infinitely many discrete spectra when both factors in \(G'\) are non-compact, they also conjecture the form of the fall discrete spectrum.

Reviewer: Fuliu Zhu (Hubei)

### MSC:

22E45 | Representations of Lie and linear algebraic groups over real fields: analytic methods |

22E46 | Semisimple Lie groups and their representations |

53A30 | Conformal differential geometry (MSC2010) |

### Citations:

Zbl 1046.22004
PDF
BibTeX
XML
Cite

\textit{T. Kobayashi} and \textit{B. Ørsted}, Adv. Math. 180, No. 2, 513--550 (2003; Zbl 1049.22006)

### References:

[1] | Binegar, B.; Zierau, R., Unitarization of a singular representation of SO(p,q), Comm. math. phys., 138, 245-258, (1991) · Zbl 0748.22009 |

[2] | A. Erdélyi, Higher Transcendental Functions, Vol. I, McGraw-Hill, New York, 1953. |

[3] | A. Erdélyi, Tables of Integral Transforms, Vol. II, McGraw-Hill, New York, 1954. |

[4] | Faraut, J., Distributions sphériques sur LES espaces hyperboliques, J. math. pures appl., 58, 369-444, (1979) · Zbl 0436.43011 |

[5] | Howe, R.; Tan, E., Homogeneous functions on light cones, Bull. amer. math. soc., 28, 1-74, (1993) · Zbl 0794.22012 |

[6] | T. Kobayashi, Singular Unitary Representations and Discrete Series for Indefinite Stiefel Manifolds \(U(p,q;F)/U(p−m,q;F)\), Mem. Amer. Math. Soc., Vol. 462, Amer. Math. Soc., Providence, RI, 1992. · Zbl 0752.22007 |

[7] | Kobayashi, T., Discrete decomposability of the restriction of \(Aq(λ)\) with respect to reductive subgroups and its applications, Invent. math., 117, 181-205, (1994) · Zbl 0826.22015 |

[8] | T. Kobayashi, Multiplicity free branching laws for unitary highest weight modules, in: K. Mimachi (Ed.), Proceedings of the Symposium on Representation Theory held at Saga, Kyushu 1997, 1997 pp. 9-17. |

[9] | Kobayashi, T., Discrete decomposability of the restriction of \(Aq(λ)\) with respect to reductive subgroups II—micro-local analysis and asymptotic K-support, Ann. math., 147, 709-729, (1998) · Zbl 0910.22016 |

[10] | Kobayashi, T., Discrete decomposability of the restriction of \(Aq(λ)\) with respect to reductive subgroups III—restriction of harish-chandra modules and associated varieties, Invent. math., 131, 229-256, (1998) · Zbl 0907.22016 |

[11] | Kobayashi, T.; Ørsted, B., Conformal geometry and branching laws for unitary representations attached to minimal nilpotent orbits, C. R. acad. sci. Paris, 326, 925-930, (1998) · Zbl 0910.22010 |

[12] | T. Kobayashi, Multiplicity-free theorem in branching problems of unitary highest weight modules, preprint. |

[13] | T. Kobayashi, Discretely decomposable restrictions of unitary representations of reductive Lie groups, in: T. Kobayashi et al. (Eds.), Analysis on Homogeneous Spaces and Representation Theory of Lie Groups, Vol. 26, 2000, pp. 98-126. |

[14] | T. Kobayashi, Branching laws of O(p,q) associated to minimal elliptic orbits, in preparation. |

[15] | T. Kobayashi, B. Ørsted, Analysis on the minimal representation of O(p,q)—I, Realization via conformal geometry, to appear in Adv. in Math. · Zbl 1046.22004 |

[16] | T.H. Koornwinder, Jacobi functions and analysis on noncompact semisimple Lie groups, Special functions: Group theoretical aspects and applications, Math. Appl. (1984) 1-85. |

[17] | B. Kostant, The vanishing scalar curvature and the minimal unitary representation of SO(4,4) in: A. Connes et al. (Eds.), Operator Algebras, Unitary Representations, Enveloping Algebras, and Invariant Theory, Vol. 92, Birkhäuser, Boston, 1990, pp. 85-124. · Zbl 0739.22012 |

[18] | Schlichtkrull, H., Eigenspaces of the Laplacian on hyperbolic spacescomposition series and integral transforms, J. funct. anal., 70, 194-219, (1987) · Zbl 0617.43005 |

[19] | D. Vogan Jr., Representations of Real Reductive Lie Groups, Progress in Math. Vol. 15, Birkhäuser, Basel, 1981. · Zbl 0469.22012 |

[20] | Vogan, D., Unitarizability of certain series of representations, Ann. math., 120, 141-187, (1984) · Zbl 0561.22010 |

[21] | D. Vogan Jr., Unitary Representations of Reductive Lie Groups, Ann. Math. Stud. Vol. 118, Princeton University Press, Princeton, 1987. · Zbl 0626.22011 |

[22] | Vogan, D., Irreducibility of discrete series representations for semisimple symmetric spaces, Adv. stud. pure math., 14, 191-221, (1988) |

[23] | H. Wong, Dolbeault cohomologies and Zuckerman modules associated with finite rank representations, Ph. D. Dissertation, Harvard University, 1992. |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.