##
**Orthogonality of Hermite polynomials in superspace and Mehler type formulae.**
*(English)*
Zbl 1233.33004

It is well known that solving the Schrödinger equation (harmonic oscillator in \(\mathbb{R}^m\) with \(O(m)\)-symmetry)
\[
-{\nabla^2\over 2}\psi+{r^2\over 2}\psi=E\psi
\]
with cartesian coordinates leads to functions written as products of one-dimensional Hermite polynomials of degrees \(k_1, \ldots,k_m\) and a factor \(e^{-r^2/2}\) for \(E={m\over 2}+\sum_{i=1}^m\,k_i\), while spherical coordinates lead to solutions \(L_j^{m/2+k-1}(r^2)H_k^{(l)}e^{ -r^2/2},\;E={m\over 2}+(2j+k)\), with \(L_{\alpha}^{\beta}\) the generalized Laguerre polynomials and \(H_k^{(l)}\) a basis for the space of spherical harmonics of degree \(k\).

Restriction of the \(O(m)\)-symmetry to a finite reflection subgroup \({\mathcal G}\), the related quantum system is of Calogero-Moser-Sutherland type given by \[ -{\Delta_ {\kappa}\over 2}\psi+{r^2\over 2}\psi=E\psi, \] with \(\Delta_{\kappa}\) the Dunkl Laplacian related to \(\mathcal G\), and again two approaches can be used: of generalized cartesian coordinates (Rösler) or generalized spherical Hermite polynomials (Dunkl setting).

The paper under review now turns its attention towards symmetry according to the symplectic group \(Sp(2n)\) and formulates the problem in a Grassmann algebra, again leading to two different bases for the algebra, which are orthogonal with respect to an inner product using the Berezin integral (see the sections 4 and 5).

Finally, the situation is studied for a full superspace with symmetry \(O(m)\times Sp(2n)\). Now the situation changes dramatically:

The main aim of the paper now becomes the following:

The results are given separately in section 5 and all results are summarized in section 6. Moreover, two detailed tables are given, stating the differences and analogies of the different types of symmetry appearing in the paper: not only an excellent sketch of the results known up to now, but also a valuable starting point for directions for further results.

Restriction of the \(O(m)\)-symmetry to a finite reflection subgroup \({\mathcal G}\), the related quantum system is of Calogero-Moser-Sutherland type given by \[ -{\Delta_ {\kappa}\over 2}\psi+{r^2\over 2}\psi=E\psi, \] with \(\Delta_{\kappa}\) the Dunkl Laplacian related to \(\mathcal G\), and again two approaches can be used: of generalized cartesian coordinates (Rösler) or generalized spherical Hermite polynomials (Dunkl setting).

The paper under review now turns its attention towards symmetry according to the symplectic group \(Sp(2n)\) and formulates the problem in a Grassmann algebra, again leading to two different bases for the algebra, which are orthogonal with respect to an inner product using the Berezin integral (see the sections 4 and 5).

Finally, the situation is studied for a full superspace with symmetry \(O(m)\times Sp(2n)\). Now the situation changes dramatically:

- –
- the spherical Hermite polynomials are not orthogonal w.r.t. the canonical inner product,
- –
- Schrödinger operators are not self-adjoint w.r.t. this inner product,
- –
- no \(O(m)\times Sp(2n)\)-invariant Mehler formula for the spherical Hermite polynomials yet.

The main aim of the paper now becomes the following:

- 1.
- construct a new inner product in the full superspace to ensure orthogonality for the spherical Hermite polynomials,
- 2.
- restore the self-adjointness of a class of Schrödinger operators of anharmonic type,
- 3.
- find an \(O(m)\times Sp(2n)\)-invariant Mehler formula.

The results are given separately in section 5 and all results are summarized in section 6. Moreover, two detailed tables are given, stating the differences and analogies of the different types of symmetry appearing in the paper: not only an excellent sketch of the results known up to now, but also a valuable starting point for directions for further results.

Reviewer: Marcel G. de Bruin (Haarlem)

### MSC:

33C45 | Orthogonal polynomials and functions of hypergeometric type (Jacobi, Laguerre, Hermite, Askey scheme, etc.) |

58C50 | Analysis on supermanifolds or graded manifolds |

42B10 | Fourier and Fourier-Stieltjes transforms and other transforms of Fourier type |