## Orthogonality of Hermite polynomials in superspace and Mehler type formulae.(English)Zbl 1233.33004

It is well known that solving the Schrödinger equation (harmonic oscillator in $$\mathbb{R}^m$$ with $$O(m)$$-symmetry) $-{\nabla^2\over 2}\psi+{r^2\over 2}\psi=E\psi$ with cartesian coordinates leads to functions written as products of one-dimensional Hermite polynomials of degrees $$k_1, \ldots,k_m$$ and a factor $$e^{-r^2/2}$$ for $$E={m\over 2}+\sum_{i=1}^m\,k_i$$, while spherical coordinates lead to solutions $$L_j^{m/2+k-1}(r^2)H_k^{(l)}e^{ -r^2/2},\;E={m\over 2}+(2j+k)$$, with $$L_{\alpha}^{\beta}$$ the generalized Laguerre polynomials and $$H_k^{(l)}$$ a basis for the space of spherical harmonics of degree $$k$$.

Restriction of the $$O(m)$$-symmetry to a finite reflection subgroup $${\mathcal G}$$, the related quantum system is of Calogero-Moser-Sutherland type given by $-{\Delta_ {\kappa}\over 2}\psi+{r^2\over 2}\psi=E\psi,$ with $$\Delta_{\kappa}$$ the Dunkl Laplacian related to $$\mathcal G$$, and again two approaches can be used: of generalized cartesian coordinates (Rösler) or generalized spherical Hermite polynomials (Dunkl setting).
The paper under review now turns its attention towards symmetry according to the symplectic group $$Sp(2n)$$ and formulates the problem in a Grassmann algebra, again leading to two different bases for the algebra, which are orthogonal with respect to an inner product using the Berezin integral (see the sections 4 and 5).
Finally, the situation is studied for a full superspace with symmetry $$O(m)\times Sp(2n)$$. Now the situation changes dramatically:
the spherical Hermite polynomials are not orthogonal w.r.t. the canonical inner product,
Schrödinger operators are not self-adjoint w.r.t. this inner product,
no $$O(m)\times Sp(2n)$$-invariant Mehler formula for the spherical Hermite polynomials yet.

The main aim of the paper now becomes the following:
1.
construct a new inner product in the full superspace to ensure orthogonality for the spherical Hermite polynomials,
2.
restore the self-adjointness of a class of Schrödinger operators of anharmonic type,
3.
find an $$O(m)\times Sp(2n)$$-invariant Mehler formula.

The results are given separately in section 5 and all results are summarized in section 6. Moreover, two detailed tables are given, stating the differences and analogies of the different types of symmetry appearing in the paper: not only an excellent sketch of the results known up to now, but also a valuable starting point for directions for further results.

### MSC:

 33C45 Orthogonal polynomials and functions of hypergeometric type (Jacobi, Laguerre, Hermite, Askey scheme, etc.) 58C50 Analysis on supermanifolds or graded manifolds 42B10 Fourier and Fourier-Stieltjes transforms and other transforms of Fourier type
Full Text: