Generalized Whittaker quotients of Schwartz functions on G-spaces

Dmitry Gourevitch, Eitan Sayag

Published 2020-01-31Version 1

Let $G$ be a reductive group over a local field $F$ of characteristic zero, Archimedean or not. Let $X$ be a $G$-space. In this paper we study the existence of generalized Whittaker quotients for the space of Schwartz functions on $X$, considered as a representation of $G$. We show that the set of nilpotent elements of the dual space to the Lie algebra such that the corresponding generalized Whittaker quotient does not vanish contains the nilpotent part of the image of the moment map, and lies in the closure of this image. This generalizes recent results of Prasad and Sakellaridis. Applying our theorems to symmetric pairs $(G,H)$ we show that there exists an infinite-dimensional $H$-distinguished representation of $G$ if and only if the real reductive group corresponding to the pair $(G,H)$ is non-compact. For quasi-split $G$ we also extend to the Archimedean case the theorem of Prasad stating that there exists a generic $H$-distinguished representation of $G$ if and only if the real reductive group corresponding to the pair $(G,H)$ is quasi-split. In the non-Archimedean case our result also gives bounds on the wave-front sets of distinguished representations.