HORN PROBLEM FOR QUASI-HERMITIAN LIE GROUPS

Abstract

In this paper, we prove some convexity results associated to orbit projection of noncompact real reductive Lie groups.

1 Introduction

This paper is concerned with convexity properties associated to orbit projection.

Let us consider two Lie groups $G\subset \tilde {G}$ with Lie algebras $\mathfrak {g}\subset \tilde {\mathfrak {g}}$ and corresponding projection $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}: \tilde {\mathfrak {g}}^{*}\to \mathfrak {g}^{*}$ . A longstanding problem has been to understand how a coadjoint orbit $\tilde {\mathcal {O}}\subset \tilde {\mathfrak {g}}^{*}$ decomposes under the projection $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}$ . For this purpose, we may define

$$ \begin{align*}\Delta_{G}(\tilde{\mathcal{O}})=\{\mathcal{O}\in \mathfrak{g}^{*}/G\,; \ \mathcal{O}\subset \pi_{\mathfrak{g},\tilde{\mathfrak{g}}}(\tilde{\mathcal{O}})\}. \end{align*} $$

When the Lie group G is compact and connected, the set $\mathfrak {g}^{*}/G$ admits a natural identification with a Weyl chamber $\mathfrak {t}^{*}_{\geq 0}$ . In this context, we have the well-known convexity theorem [12, 1, 10, 16, 13, 35, 22].

Theorem 1.1. Suppose that G is compact connected and that the projection $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}$ is proper when restricted to $\tilde {\mathcal {O}}$ . Then $\Delta _{G}(\tilde {\mathcal {O}})=\{\xi \in \mathfrak {t}^{*}_{\geq 0}\,; \ G\xi \subset \pi _{\mathfrak {g},\tilde {\mathfrak {g}}}(\tilde {\mathcal {O}})\}$ is a closed convex locally polyhedral subset of $\mathfrak {t}^{*}$ .

When the Lie group $\tilde {G}$ is also compact and connected, we may consider

(1) $$ \begin{align} \Delta(\tilde{G},G):=\left\{(\tilde{\xi},\xi)\in \tilde{\mathfrak{t}}^{*}_{\geq 0}\times\mathfrak{t}^{*}_{\geq 0}; \ G\xi\,\subset \,\pi_{\mathfrak{g},\tilde{\mathfrak{g}}}\big(\tilde{G}\tilde{\xi}\,\big)\right\}. \end{align} $$

Here is another convexity theorem [14, 17, 4, 2, 3, 25, 19, 20, 36].

Theorem 1.2. Suppose that $G\subset \tilde {G}$ are compact connected Lie groups. Then $\Delta (\tilde {G},G)$ is a closed convex polyhedral cone and we can parametrize its facets by cohomological means (i.e., Schubert calculus).

In this article, we obtain an extension of Theorems 1.1 and 1.2 in a case where G and $\tilde {G}$ are both noncompact real reductive Lie groups.

Let us explain what framework we are considering. Let $\tilde {K}$ be a maximal compact subgroup of $\tilde {G}$ . We suppose that $\tilde {G}/\tilde {K}$ is a Hermitian symmetric space of a noncompact type. Among the elliptic coadjoint orbits of $\tilde {G}$ , some of them are naturally Kähler $\tilde {K}$ -manifolds. These orbits are called the holomorphic coadjoint orbits of $\tilde {G}$ . They are the strongly elliptic coadjoint orbits closely related to the holomorphic discrete series of Harish–Chandra. These orbits intersect the Weyl chamber $\tilde {\mathfrak {t}}^{*}_{\geq 0}$ of $\tilde {K}$ into a subchamber $\tilde {\mathcal {C}}_{\mathrm {hol}}$ called the holomorphic chamber. The basic fact here is that the union

$$ \begin{align*}\mathcal{C}_{\tilde{G}/\tilde{K}}^{0}:=\bigcup_{\tilde{a}\in\tilde{\mathcal{C}}_{\mathrm{hol}}} \tilde{G}\tilde{a} \end{align*} $$

is an open invariant convex cone of $\tilde {\mathfrak {g}}^{*}$ . See §2.1 for more details.

In this article, we work in the context where $\tilde {G}/\tilde {K}$ admits a sub-Hermitian symmetric space of a noncompact type $G/K$ . For the convenience of the reader, we list below some examples of the pairs $(\tilde {G},G)$ :

As the projection $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}: \tilde {\mathfrak {g}}^{*}\to \mathfrak {g}^{*}$ sends the convex cone $\mathcal {C}_{\tilde {G}/\tilde {K}}^{0}$ inside the convex cone $\mathcal {C}_{G/K}^{0}$ , it is natural to study the following object reminiscent of equation (1):

(2) $$ \begin{align} \Delta_{\mathrm{hol}}(\tilde{G},G):=\left\{(\tilde{\xi},\xi)\in \tilde{\mathcal{C}}_{\mathrm{hol}}\times\mathcal{C}_{\mathrm{hol}}; \ G\xi\,\subset \,\pi_{\mathfrak{g},\tilde{\mathfrak{g}}}\big(\tilde{G}\tilde{\xi}\,\big)\right\}. \end{align} $$

Let $\tilde {\mu }\in \tilde {\mathcal {C}}_{\mathrm {hol}}$ . We will also give a particular attention to the intersection of $\Delta _{\mathrm {hol}}(\tilde {G},G)$ with the linear subspace $\tilde {\xi }=\tilde {\mu }$ , that is to say

(3) $$ \begin{align} \Delta_{G}(\tilde{G}\tilde{\mu}):=\left\{\xi\in \mathcal{C}_{\mathrm{hol}}; \ G\xi\,\subset \,\pi_{\mathfrak{g},\tilde{\mathfrak{g}}}\big(\tilde{G}\tilde{\mu}\,\big)\right\}. \end{align} $$

Consider the case where G is embedded diagonally in $\tilde {G}:=G^{s}$ for $s\geq 2$ . The corresponding set $\Delta _{\mathrm {hol}}(G^{s},G)$ is called the holomorphic Horn cone, and it is defined as follows:

$$ \begin{align*}\mathrm{Horn}_{\mathrm{hol}}^{s}(G):=\Big\{(\xi_{1},\cdots,\xi_{s+1})\in \mathcal{C}_{\mathrm{hol}}^{s+1}; \ G\xi_{s+1}\,\subset \sum_{j=1}^{s} G\xi_{j}\Big\}. \end{align*} $$

The first result of this article is the following theorem.

Theorem A.

• • $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is a closed convex cone of $\tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ .

• • $\mathrm {Horn}_{\mathrm {hol}}^{s}(G)$ is a closed convex cone of $\mathcal {C}_{\mathrm {hol}}^{s+1}$ for any $s\geq 2$ .

We obtain the following corollary which corresponds to a result of A. Weinstein [38].

Corollary. For any $\tilde {\mu }\in \tilde {\mathcal {C}}_{\mathrm { hol}}$ , $\Delta _{G}(\tilde {G}\tilde {\mu })$ is a closed and convex subset of $\mathcal {C}_{\mathrm { hol}}$ .

A first description of the closed convex cone $\Delta _{\mathrm {hol}}(\tilde {G},G)$ goes as follows. The quotient $\mathfrak {q}$ of the tangent spaces $\textbf {T}_{e}G/K$ and $\textbf { T}_{e}\tilde {G}/\tilde {K}$ has a natural structure of a Hermitian K-vector space. The symmetric algebra $\mathrm {Sym}(\mathfrak {q})$ of $\mathfrak {q}$ defines an admissible K-module. The irreducible representations of K (resp. $\tilde {K}$ ) are parametrized by a semi-group $\wedge ^{*}_{+}$ (resp. $\tilde {\wedge }^{*}_{+}$ ). For any $\lambda \in \wedge ^{*}_{+}$ (resp. $\tilde {\lambda }\in \tilde {\wedge }^{*}_{+}$ ), we denote by $V_{\lambda }^{K}$ (resp. $V_{\tilde {\lambda }}^{\tilde {K}}$ ) the irreducible representation of K (resp. $\tilde {K}$ ) with highest weight $\lambda $ (resp. $\tilde {\lambda }$ ). If E is a representation of K, we denote by $\left [V^{K}_{\lambda } : E\right ]$ the multiplicity of $V^{K}_{\lambda }$ in E.

Definition 1.3.

1. 1. $\Pi ^{\mathbb {Z}}_{\mathfrak {q}}(\tilde {K},K)$ is the semigroup of $\tilde {\wedge }^{*}_{+}\times \wedge ^{*}_{+}$ defined by the conditions:

   $$ \begin{align*}(\tilde{\lambda},\lambda)\in \Pi^{\mathbb{Z}}_{\mathfrak{q}}(\tilde{K},K)\quad \Longleftrightarrow \quad\Big[V^{K}_{\lambda}\,:\,V_{\tilde{\lambda}}^{\tilde{K}}\otimes \mathrm{Sym}(\mathfrak{q})\Big]\neq 0. \end{align*} $$

2. 2. $\Pi _{\mathfrak {q}}(\tilde {K},K)$ is the convex cone defined as the closure of $\mathbb {Q}^{>0}\cdot \Pi ^{\mathbb {Z}}_{\mathfrak {q}}(\tilde {K},K)$ .

The second result of this article is the following theorem.

Theorem B. We have the equality

(4) $$ \begin{align} \Delta_{\mathrm{ hol}}(\tilde{G},G)=\Pi_{\mathfrak{q}}(\tilde{K},K)\, \bigcap\, \tilde{\mathcal{C}}_{\mathrm{ hol}}\!\times\!\mathcal{C}_{\mathrm{ hol}}. \end{align} $$

A natural question is the description of the facets of the convex cone $\Delta _{\mathrm {hol}}(\tilde {G},G)$ . In order to do that, we consider the group $\tilde {K}$ endowed with the following $\tilde {K}\times K$ -action: $(\tilde {k},k)\cdot \tilde {a}=\tilde {k}\tilde {a}k^{-1}$ . The cotangent space $\textbf { T}^{*}\tilde {K}$ is then a symplectic manifold equipped with a Hamiltonian action of $\tilde {K}\times K$ . We consider now the Hamiltonian $\tilde {K}\times K$ -manifold $\textbf { T}^{*}\tilde {K}\times \mathfrak {q}$ , and we denote by $\Delta (\textbf{T}^{*}\tilde {K}\times \mathfrak {q})$ the corresponding Kirwan polyhedron.

Let $W=N(T)/T$ be the Weyl group of $(K,T)$ , and let $w_{0}$ be the longest Weyl group element. Define an involution $*:\mathfrak {t}^{*}\to \mathfrak {t}^{*}$ by $\xi ^{*}=- w_{0}\xi $ . A standard result permits to affirm that $(\tilde {\xi },\xi )\in \Pi _{\mathfrak {q}}(\tilde {K},K)$ if and only if $(\tilde {\xi },\xi ^{*})\in \Delta (\textbf{T}^{*}\tilde {K}\times \mathfrak {q})$ (see §4.2).

We obtain then another version of Theorem B.

Theorem B, second version. An element $(\tilde {\xi },\xi )$ belongs to $\Delta _{\mathrm { hol}}(\tilde {G},G)$ if and only if

$$ \begin{align*}(\tilde{\xi},\xi)\in\tilde{\mathcal{C}}_{\mathrm{ hol}}\times\mathcal{C}_{\mathrm{ hol}}\quad \mathrm{ and}\quad (\tilde{\xi},\xi^{*})\in\Delta(\textbf{T}^{*}\tilde{K}\times \mathfrak{q}). \end{align*} $$

Thanks to the second version of Theorem B, a natural way to describe the facets of the cone $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is to exhibit those of the Kirwan polyhedron $\Delta (\textbf{T}^{*}\tilde {K}\times \mathfrak {q})$ . In this later case, it can be done using Ressayre’s data (see §5).

The second version of Theorem B permits also the following description of the convex subsets $\Delta _{G}(\tilde {G}\tilde {\mu })$ , $\tilde {\mu }\in \tilde {\mathcal {C}}_{\mathrm {hol}}$ . Let $\Delta _{K}(\tilde {K}\tilde {\mu }\times \overline {\mathfrak {q}})$ be the Kirwan polyhedron associated to the Hamiltonian action of K on $\tilde {K}\tilde {\mu }\times \overline {\mathfrak {q}}$ , where $\overline {\mathfrak {q}}$ denotes the K-module $\mathfrak {q}$ with opposite complex structure.

Theorem C. For any $\tilde {\mu }\in \tilde {\mathcal {C}}_{\mathrm { hol}}$ , we have $\Delta _{G}(\tilde {G}\tilde {\mu })=\Delta _{K}(\tilde {K}\tilde {\mu }\times \overline {\mathfrak {q}})$ .

Let us detail Theorem C in the case where G is embedded in $\tilde {G}=G\times G$ diagonally. We denote by $\mathfrak {p}$ the K-Hermitian space $\textbf { T}_{e}G/K$ . Let $\kappa $ be the Killing form of the Lie algebra $\mathfrak {g}$ . The vector space $\overline {\mathfrak {p}}$ is equipped with the symplectic $2$ -form $\Omega _{\bar {\mathfrak {p}}}(X,Y)=-\kappa (z,[X,Y])$ and the compatible complex structure $-\mathrm {ad}(z)$ .

Let us denote by $\Delta _{K}(K\mu _{1}\times K\mu _{2}\times \overline {\mathfrak {p}})$ and by $\Delta _{K}(\overline {\mathfrak {p}})$ the Kirwan polyhedrons relative to the Hamiltonian actions of K on $K\mu _{1}\times K\mu _{2}\times \overline {\mathfrak {p}}$ and on $\overline {\mathfrak {p}}$ . Theorem C says that, for any $\mu _{1},\mu _{2}\in \mathcal {C}_{\mathrm {hol}}$ , the convex set $\Delta _{G}(G\mu _{1}\times G\mu _{2})$ is equal to the Kirwan polyhedron $\Delta _{K}(K\mu _{1}\times K\mu _{2}\times \overline {\mathfrak {p}})$ .

To any nonempty subset $\mathcal {C}$ of a real vector space E, we may associate its asymptotic cone $\mathrm {As}(\mathcal {C}) \subset E$ which is the set formed by the limits $y=\lim _{k\to \infty } t_{k} y_{k}$ , where $(t_{k})$ is a sequence of nonnegative reals converging to $0$ and $y_{k}\in \mathcal {C}$ .

We finally get the following description of the asymptotic cone of $\Delta _{G}(G\mu _{1}\times G\mu _{2})$ .

Corollary D. For any $\mu _{1},\mu _{2}\in \mathcal {C}_{\mathrm {hol}}$ , the asymptotic cone of $\Delta _{G}(G\mu _{1}\times G\mu _{2})$ is equal to $\Delta _{K}(\overline {\mathfrak {p}})$ .

In [29] §5, we explained how to describe the cone $\Delta _{K}(\overline {\mathfrak {p}})$ in terms of strongly orthogonal roots.

Let us finish this introduction with few remarks on related works:

1. – When G is compact, equal to the maximal compact subgroup $\tilde {K}$ of $\tilde {G}$ , the results of Theorems B and C were already obtained by G. Deltour in his thesis [6, 7]. He proved the equality $\Delta _{\tilde {K}}(\tilde {G}\tilde {\mu })=\Delta _{\tilde {K}}(\tilde {K}\tilde {\mu }\times \overline {\tilde {\mathfrak {p}}})$ by showing that the coadjoint orbit $\tilde {G}\tilde {\mu }$ admits a $\tilde {K}$ -equivariant symplectomorphism with $\tilde {K}\tilde {\mu }\times \overline {\tilde {\mathfrak {p}}}$ , thus generalizing an earlier result of D. McDuff [26]. We explain in §7 a conjectural symplectomorphism that would lead to the relation $\Delta _{G}(\tilde {G}\tilde {\mu })=\Delta _{K}(\tilde {K}\tilde {\mu }\times \overline {\mathfrak {q}})$ .

2. – In [9], A. Eshmatov and P. Foth proposed a description of the set $\Delta _{G}(G\mu _{1}\times G\mu _{2})$ . But their computations do not give the same result as ours. From their main result (Theorem 3.2), it follows that the asymptotic cone of $\Delta _{G}(G\mu _{1}\times G\mu _{2})$ is equal to the intersection of the Kirwan polyhedron $\Delta _{T}(\overline {\mathfrak {p}})$ with the Weyl chamber $\mathfrak {t}^{*}_{\geq 0}$ . But since $\Delta _{K}(\overline {\mathfrak {p}})\neq \Delta _{T}(\overline {\mathfrak {p}})\cap \mathfrak {t}^{*}_{\geq 0}$ in general, it is in contradiction with Corollary D.

Notations

In this paper, we take the convention of A. Knapp [18]: A connected real reductive Lie group G means that we have a Cartan involution $\Theta $ on G such that the fixed point set $K:=G^{\Theta }$ is a connected maximal compact subgroup. We have Cartan decompositions at the level of Lie algebras $\mathfrak {g}=\mathfrak {k}\oplus \mathfrak {p}$ and at the level of the group $G\simeq K\times \exp (\mathfrak {p})$ . We denote by b a G-invariant nondegenerate bilinear form on $\mathfrak {g}$ that is equal to the Killing form on $[\mathfrak {g},\mathfrak {g}]$ , and that defines a K-invariant scalar product $(X,Y):=-b(X,\Theta (Y))$ . We will use the K-equivariant identification $\xi \mapsto \tilde {\xi },\ \mathfrak {g}^{*}\simeq \mathfrak {g}$ defined by $(\tilde {\xi },X):=\langle \xi ,X\rangle $ for $\xi \in \mathfrak {g}^{*}$ and $X\in \mathfrak {g}$ .

When a Lie group H acts on a manifold N, the stabilizer subgroup of $n\in N$ is denoted by $H_{n}=\{g\in G,gn=n\}$ and its Lie algebra by $\mathfrak {h}_{n}$ . Let us define

(5) $$ \begin{align} \dim_{H}(\mathcal{X})=\min_{n\in \mathcal{X}} \dim(\mathfrak{h}_{n}) \end{align} $$

for any subset $\mathcal {X}\subset N$ .

2 The cone $\Delta _{\mathrm {hol}}(\tilde {G},G)$ : first properties

We assume here that $G/K$ is a Hermitian symmetric space of a noncompact type, that is to say, there exists a G-invariant complex structure on the manifold $G/K$ or, equivalently, there exists a K-invariant element $z\in \mathfrak {k}$ such that $\mathrm {ad}(z)\vert _{\mathfrak {p}}$ defines a complex structure on $\mathfrak {p}$ : $(\mathrm {ad}(z)\vert _{\mathfrak {p}})^{2}= -\mathrm {Id}_{\mathfrak {p}}$ . This condition imposes that the ranks of G and K are equal.

We are interested in the following closed invariant convex cone of $\mathfrak {g}^{*}$ :

$$ \begin{align*}\mathcal{C}_{G/K}=\left\{\xi\in\mathfrak{g}^{*},\langle \xi,g z\rangle \geq 0,\ \forall g\in G\right\}. \end{align*} $$

2.1 The holomorphic chamber

Let T be a maximal torus of K, with Lie algebra $\mathfrak {t}$ . Its dual $\mathfrak {t}^{*}$ can be seen as the subspace of $\mathfrak {g}^{*}$ fixed by T. Let us denote by $\mathfrak {g}^{*}_{e}$ the set formed by the elliptic elements: In other words, $\mathfrak {g}^{*}_{e}:=\mathrm {Ad}^{*}(G)\cdot \mathfrak {t}^{*}$ .

Following [38], we consider the invariant open subset $\mathfrak {g}^{*}_{se}=\{\xi \in \mathfrak {g}^{*}\,\vert \, G_{\xi } \ \mathrm {is\ compact} \}$ of strongly elliptic elements. It is nonempty since the groups G and K have the same rank.

We start with the following basic facts.

Lemma 2.1.

• • $\mathfrak {g}^{*}_{se}$ is contained in $\mathfrak {g}^{*}_{e}$ .

• • The interior $\mathcal {C}_{G/K}^{0}$ of the cone $\mathcal {C}_{G/K}$ is contained in $\mathfrak {g}^{*}_{se}$ .

Proof. The first point is due to the fact that every compact subgroup of G is conjugate to a subgroup of K.

Let $\xi \in \mathcal {C}_{G/K}^{0}$ . There exists $\epsilon>0$ so that

$$ \begin{align*}\langle \xi +\eta,g z\rangle \geq 0,\quad \forall g\in G,\quad \forall \|\eta\|\leq \epsilon. \end{align*} $$

It implies that $|\langle \eta ,g z\rangle | \leq \langle \xi , z\rangle $ , $\forall g\in G_{\xi }$ and $\forall \|\eta \|\leq \epsilon $ . In other words, the adjoint orbit $G_{\xi }\cdot z\subset \mathfrak {g}$ is bounded. For any $g=e^{X} k$ , with $(X,k)\in \mathfrak {p}\times K$ , a direct computation shows that $\|gz\|=\|e^{X} z\|\geq \|[z,X]\|=\|X\|$ . Then, there exists $\rho>0$ such that $\|X\|\leq \rho $ if $e^{X} k\in G_{\xi }$ for some $k\in K$ . It follows that the stabilizer subgroup $G_{\xi }$ is compact.

Let $\wedge ^{*}\subset \mathfrak {t}^{*}$ be the weight lattice: By definition, $\alpha \in \wedge ^{*}$ if and only if $i\alpha $ is the differential of a character of T. Let $\mathfrak {R}\subset \wedge ^{*}$ be the set of roots for the action of T on $\mathfrak {g}\otimes \mathbb {C}$ . We have $\mathfrak {R}=\mathfrak {R}_{c}\cup \mathfrak {R}_{n}$ , where $\mathfrak {R}_{c}$ and $\mathfrak {R}_{n}$ are, respectively, the set of roots for the action of T on $\mathfrak {k}\otimes \mathbb {C}$ and $\mathfrak {p}\otimes \mathbb {C}$ . We fix a system of positive roots $\mathfrak {R}^{+}_{c}$ in $\mathfrak {R}_{c}$ , and we denote by $\mathfrak {t}^{*}_{\geq 0}$ the corresponding Weyl chamber.

We have $\mathfrak {p}\otimes \mathbb {C}=\mathfrak {p}^{+}\oplus \mathfrak {p}^{-}$ , where the K-module $\mathfrak {p}^{\pm }$ is equal to $\ker (\mathrm {ad}(z)\mp i)$ . Let $\mathfrak {R}^{\pm ,z}_{n}$ be the set of roots for the action of T on $\mathfrak {p}^{\pm }$ . The union

(6) $$ \begin{align} \mathfrak{R}^{+}=\mathfrak{R}^{+}_{c}\cup\mathfrak{R}^{+,z}_{n} \end{align} $$

defines then a system of positive roots in $\mathfrak {R}$ . We notice that $\mathfrak {R}^{+,z}_{n}$ is the set of roots $\beta \in \mathfrak {R}$ satisfying $\langle \beta , z\rangle =1$ . Hence, $\mathfrak {R}^{+,z}_{n}$ is invariant relatively to the action of the Weyl group $W=N(T)/T$ .

Let us recall the following classical fact concerning the parametrization of the G-orbits in $\mathcal {C}_{G/K}^{0}$ via the holomorphic chamber

$$ \begin{align*}\mathcal{C}_{\mathrm{hol}}:=\{\xi\in\mathfrak{t}^{*}_{\geq 0},(\xi,\beta)>0,\ \forall \beta\in \mathfrak{R}^{+,z}_{n}\}. \end{align*} $$

The elliptic coadjoint orbits of G, i.e., those contained in $\mathfrak {g}^{*}_{e}$ , are parameterized by the Weyl chamber $\mathfrak {t}^{*}_{\geq 0}$ . Thus, we have a projection $\mathrm {p}: \mathfrak {g}^{*}_{e}\to \mathfrak {t}^{*}_{\geq 0}$ , defined by the relations $G\xi \cap \mathfrak {t}^{*}_{\geq 0}=\{ \mathrm {p}(\xi )\}$ , and that induces a bijection $\mathfrak {g}^{*}_{e}/G\simeq \mathfrak {t}^{*}_{\geq 0}$ .

Proposition 2.2. The set $\mathrm {p}(\mathcal {C}_{G/K}^{0})$ is equal to $\mathcal {C}_{\mathrm {hol}}$ . In other words, the map $\mathrm {p}$ induces a bijective map between the set of G-orbits in $\mathcal {C}_{G/K}^{0}$ and the holomorphic chamber $\mathcal {C}_{\mathrm {hol}}$ .

Proof. Let us first prove that $\mathrm {p}(\mathcal {C}_{G/K}^{0})=\mathfrak {t}^{*}_{\geq 0}\cap \mathcal {C}_{G/K}^{0}$ is contained in $\mathcal {C}_{\mathrm {hol}}$ . Let $\xi \in \mathfrak {t}^{*}_{\geq 0}\cap \mathcal {C}_{G/K}^{0}$ : We have to check that $(\xi ,\beta )>0$ for any $\beta \in \mathfrak {R}^{+,z}_{n}$ . Let $X_{\beta },Y_{\beta }\in \mathfrak {p}$ such that $X_{\beta }+i Y_{\beta }\in (\mathfrak {p}\otimes \mathbb {C})_{\beta }$ . We choose the following normalization: The vector $h_{\beta }:=[X_{\beta },Y_{\beta }]$ satisfies $\langle \beta ,h_{\beta }\rangle =1$ . We see then that $(\xi ,\beta )=\frac {1}{\|h_{\beta }\|^{2}}\langle \xi ,h_{\beta }\rangle $ for any $\xi \in \mathfrak {g}^{*}$ . Standard computation [28] gives: $e^{t\,\mathrm {ad}(X_{\beta })}z=z +(\mathrm {cosh}(t)-1)h_{\beta } +\mathrm {sinh}(t)Y_{\beta },\ \forall t\in \mathbb {R}$ . By definition, we must have $\langle \xi +\eta ,e^{t\,\mathrm {ad}(X_{\beta })} z\rangle \geq 0, \forall t\in \mathbb {R}$ , for any $\eta \in \mathfrak {t}^{*}$ small enough. It imposes that $\langle \xi ,h_{\beta }\rangle>0$ . The first point is settled.

The other inclusion $\mathcal {C}_{\mathrm {hol}}\subset \mathfrak {t}^{*}_{\geq 0}\cap \mathcal {C}_{G/K}^{0}$ is a consequence of the next lemma.

Lemma 2.3. For any compact subset $\mathcal {K}$ of $\mathcal {C}_{\mathrm {hol}}$ , there exists $c_{\mathcal {K}}>0$ such that $\langle \xi ,g z\rangle \geq c_{\mathcal {K}} \|g z\|$ , $\forall g\in G$ , $\forall \xi \in \mathcal {K}$ .

Proof. Let us choose some maximal strongly orthogonal system $\Sigma \subset \mathfrak {R}^{+,z}_{n}$ . The real span $\mathfrak {a}$ of the $X_{\beta },\beta \in \Sigma $ is a maximal abelian subspace of $\mathfrak {p}$ . Hence, any element $g\in G$ can be written $g=k e^{X(t)} k^{\prime }$ with $X(t)=\sum _{\beta \in \Sigma }t_{\beta } X_{\beta }$ and $k,k^{\prime }\in K$ . We get

(7) $$ \begin{align} gz=k\left(z +\sum_{\beta\in\Sigma}(\mathrm{cosh}(t_{\beta})-1)h_{\beta} +\sum_{\beta\in\Sigma}\mathrm{sinh}(t_{\beta})Y_{\beta}\right) \end{align} $$

and

$$ \begin{align*}\langle \xi,g z\rangle=\langle k^{-1}\xi,z\rangle + \sum_{\beta\in\Sigma}(\mathrm{cosh}(t_{\beta})-1) \langle k^{-1}\xi,h_{\beta}\rangle. \end{align*} $$

For any $\xi \in \mathcal {C}_{\mathrm {hol}}$ , we define $c_{\xi } := \min _{\beta \in \mathfrak {R}^{+,z}_{n}}\langle \xi ,h_{\beta }\rangle>0$ . Let $\pi :\mathfrak {k}^{*}\to \mathfrak {t}^{*}$ be the projection. We have $\langle k^{-1}\xi ,z\rangle =\langle \pi (k^{-1}\xi ),z\rangle $ and $\langle k^{-1}\xi ,h_{\beta }\rangle =\langle \pi (k^{-1}\xi ),h_{\beta }\rangle $ . The convexity theorem of Kostant tell us that $\pi (k^{-1}\xi )$ belongs to the convex hull of $\{w\xi , w\in W\}$ . It follows that $\langle k^{-1}\xi ,z\rangle \geq \langle \xi ,z\rangle>0 \quad \mathrm {and}\quad \langle k^{-1}\xi ,h_{\beta }\rangle \geq c_{\xi }>0$ for any $k\in K$ . We obtain then that $\langle \xi ,g z\rangle \geq \frac {1}{2}\min (\langle \xi ,z\rangle , c_{\xi }) e^{\|X(t)\|}$ for any $\xi \in \mathcal {C}_{\mathrm {hol}}$ , where $\|X(t)\|=\sup _{\beta } |t_{\beta }|$ . From equation (7), it is not difficult to see that there exists $C>0$ such that $\|gz\|\leq C e^{\|X(t)\|}$ for any $g=k e^{X(t)} k^{\prime }\in G$ .

Let $\mathcal {K}$ be a compact subset of $\mathcal {C}_{\mathrm {hol}}$ . Take $c_{\mathcal {K}}=\frac {1}{2C}\min (\min _{\xi \in \mathcal {K}}\langle \xi ,z\rangle , \min _{\xi \in \mathcal {K}}c_{\xi })>0$ . The previous computations show that $\langle \xi ,g z\rangle \geq c_{\mathcal {K}} \|g z\|,\ \forall g\in G,\ \forall \xi \in \mathcal {K}$ .

The following result is needed in §4.1.

Lemma 2.4. The map $\mathrm {p}: \mathcal {C}_{G/K}^{0}\to \mathcal {C}_{\mathrm {hol}}$ is continuous.

Proof. Let $(\xi _{n})$ be a sequence of $\mathcal {C}_{G/K}^{0}$ converging to $\xi _{\infty }\in \mathcal {C}_{G/K}^{0}$ . Let $\xi ^{\prime }_{n} =\mathrm {p}(\xi _{n})$ and $\xi ^{\prime }_{\infty } =\mathrm {p}(\xi _{\infty })$ : We have to prove that the sequence $(\xi ^{\prime }_{n})$ converges to $\xi ^{\prime }_{\infty }$ . We choose elements $g_{n},g_{\infty }\in G$ such that $\xi _{n}=g_{n}\xi ^{\prime }_{n},\forall n$ and $\xi _{\infty }=g_{\infty }\xi ^{\prime }_{\infty }$ .

First, we notice that $-b(\xi _{n},\xi _{n})=\|\xi ^{\prime }_{n}\|^{2}$ ; hence, the sequence $(\xi ^{\prime }_{n})$ is bounded. We will now prove that the sequence $(g_{n})$ is bounded. Let $\epsilon>0$ such that $\langle \xi _{\infty }+\eta ,gz\rangle \geq 0$ , $\forall g\in G$ , $\forall \|\eta \|\leq \epsilon $ . If $\|\xi -\xi _{\infty }\|\leq \epsilon /2$ , we write $\xi =\frac {1}{2}(\xi _{\infty }+ 2(\xi -\xi _{\infty }))+\frac {1}{2}\xi _{\infty }$ , and then

$$ \begin{align*}\langle\xi,gz\rangle= \frac{1}{2}\langle\xi_{\infty}+ 2(\xi-\xi_{\infty}),gz\rangle +\frac{1}{2}\langle\xi_{\infty},gz\rangle\geq \frac{1}{2}\langle\xi_{\infty},gz\rangle, \quad \forall g\in G. \end{align*} $$

Now we have $\langle \xi ^{\prime }_{n},z\rangle =\langle \xi _{n},g_{n}z\rangle \geq \frac {1}{2}\langle \xi _{\infty },g_{n}z\rangle $ if n is large enough. This shows that the sequence $\langle \xi _{\infty },g_{n}z\rangle $ is bounded. If we use Lemma 2.3, we can conclude that the sequence $(g_{n})$ is bounded.

Let $(\xi ^{\prime }_{\phi (n)})$ be a subsequence converging to $\ell \in \mathfrak {t}^{*}_{\geq 0}$ . Since $(g_{\phi (n)})$ is bounded, there exists a subsequence $(g_{\phi \circ \psi (n)})$ converging to $h\in G$ . From the relations $\xi _{\phi \circ \psi (n)}=$ $g_{\phi \circ \psi (n)}\xi ^{\prime }_{\phi \circ \psi (n)},\forall n\in \mathbb {N}$ , we obtain $\xi _{\infty }=h\ell $ . Then $\ell =\mathrm {p}(\xi _{\infty })= \xi _{\infty }^{\prime }$ . Since every subsequence of $(\xi ^{\prime }_{n})$ has a subsequential limit to $\xi _{\infty }^{\prime }$ , then the sequence $(\xi ^{\prime }_{n})$ converges to $\xi ^{\prime }_{\infty }$ .

2.2 The cone $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is closed

We suppose that $G/K$ is a complex submanifold of a Hermitian symmetric space $\tilde {G}/\tilde {K}$ . In other words, $\tilde {G}$ is a reductive real Lie group such that $G\subset \tilde {G}$ is a closed connected subgroup preserved by the Cartan involution, and $\tilde {K}$ is a maximal compact subgroup of $\tilde {G}$ containing K. We denote by $\tilde {\mathfrak {g}}$ and $\tilde {\mathfrak {k}}$ the Lie algebras of $\tilde {G}$ and $\tilde {K}$ , respectively. We suppose that there exists a $\tilde {K}$ -invariant element $z\in \mathfrak {k}$ such that $\mathrm {ad}(z)\vert _{\tilde {\mathfrak {p}}}$ defines a complex structure on $\tilde {\mathfrak {p}}$ : $(\mathrm {ad}(z)\vert _{\tilde {\mathfrak {p}}})^{2}= -Id_{\tilde {\mathfrak {p}}}$ .

Let $\mathcal {C}_{\tilde {G}/\tilde {K}}\subset \tilde {\mathfrak {g}}^{*}$ be the closed invariant cone associated to the Hermitian symmetric space $\tilde {G}/\tilde {K}$ . We start with the following key fact.

Lemma 2.5. The projection $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}: \tilde {\mathfrak {g}}^{*}\to \mathfrak {g}^{*}$ sends $\mathcal {C}_{\tilde {G}/\tilde {K}}^{0}$ into $\mathcal {C}_{G/K}^{0}$ .

Proof. Let $\tilde {\xi }\in \mathcal {C}_{\tilde {G}/\tilde {K}}^{0}$ and $\xi =\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}(\tilde {\xi })$ . Then $\langle \tilde {\xi }+\tilde {\eta },\tilde {g} z\rangle \geq 0,\ \forall \tilde {g}\in \tilde {G}$ if $\tilde {\eta }\in \tilde {\mathfrak {g}}^{*}$ is small enough. It follows that $\langle \xi +\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}(\tilde {\eta }),g z\rangle =\langle \tilde {\xi }+\tilde {\eta },g z\rangle \geq 0,\ \forall g\in G$ if $\tilde {\eta }$ is small enough. Since $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}$ is an open map, we can conclude that $\xi \in \mathcal {C}_{G/K}^{0}$ .

Let $\tilde {T}$ be a maximal torus of $\tilde {K}$ , with Lie algebra $\tilde {\mathfrak {t}}$ . The $\tilde {G}$ -orbits in the interior of $\mathcal {C}_{\tilde {G}/\tilde {K}}$ are parametrized by the holomorphic chamber $\tilde {\mathcal {C}}_{\mathrm {hol}}\subset \tilde {\mathfrak {t}}^{*}$ . The previous lemma says that the projection $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}(\tilde {\mathcal {O}})$ of any $\tilde {G}$ -orbit $\tilde {\mathcal {O}}\subset \mathcal {C}_{\tilde {G}/\tilde {K}}^{0}$ is the union of G-orbits $\mathcal {O}\subset \mathcal {C}^{0}_{G/K}$ . So it is natural to study the following object:

(8) $$ \begin{align} \Delta_{\mathrm{hol}}(\tilde{G},G):=\left\{(\tilde{\xi},\xi)\in \tilde{\mathcal{C}}_{\mathrm{hol}}\times\mathcal{C}_{\mathrm{hol}}; \ G\xi\,\subset \,\pi_{\mathfrak{g},\tilde{\mathfrak{g}}}\big(\tilde{G}\tilde{\xi}\,\big)\right\}. \end{align} $$

Here is a first result.

Proposition 2.6. $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is a closed cone of $\tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ .

Proof. Suppose that a sequence $(\tilde {\xi }_{n},\xi _{n})\in \Delta _{\mathrm {hol}}(\tilde {G},G)$ converges to $(\tilde {\xi }_{\infty },\xi _{\infty })\in \tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ . By definition, there exists a sequence $(\tilde {g}_{n},g_{n})\in \tilde {G}\times G$ such that $g_{n} \xi _{n}=\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}(\tilde {g}_{n}\tilde {\xi }_{n})$ . Let $\tilde {h}_{n}:=g_{n}^{-1}\tilde {g}_{n}$ so that $\xi _{n}=\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}(\tilde {h}_{n}\tilde {\xi }_{n})$ and $\langle \tilde {h}_{n}\tilde {\xi }_{n}, z\rangle =\langle \xi _{n}, z\rangle $ . We use now that the sequence $\langle \xi _{n}, z\rangle $ is bounded and that the sequence $\tilde {\xi }_{n}$ belongs to a compact subset of $\tilde {\mathcal {C}}_{\mathrm {hol}}$ . Thanks to Lemma 2.3, these facts imply that $\|\tilde {h}_{n}^{-1}z\|$ is a bounded sequence. Hence, $\tilde {h}_{n}$ admits a subsequence converging to $\tilde {h}_{\infty }$ . So we get $\xi _{\infty }=\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}(\tilde {h}_{\infty }\tilde {\xi }_{\infty })$ , and that proves that $(\tilde {\xi }_{\infty },\xi _{\infty })\in \Delta _{\mathrm {hol}}(\tilde {G},G)$ .

2.3 Rational and weakly regular points

Let $(M,\Omega )$ be a symplectic manifold. We suppose that there exists a line bundle $\mathcal {L}$ with connection $\nabla $ that prequantizes the $2$ -form $\Omega $ : In other words, $\nabla ^{2}=- i\, \Omega $ . Let K be a compact connected Lie group acting on $\mathcal {L}\to M$ , and leaving the connection invariant. Let $\Phi _{K}:M\to \mathfrak {k}^{*}$ be the moment map defined by Kostant’s relations

(9) $$ \begin{align} L_{X}-\nabla_{X}=i\langle \Phi_{K},X\rangle,\quad \forall X\in\mathfrak{k}. \end{align} $$

Here $L_{X}$ is the Lie derivative acting on the sections of $\mathcal {L}\to M$ .

Remark that relations (9) imply, via the equivariant Bianchi formula, the relations

(10) $$ \begin{align} \iota(X_{M})\Omega=- d\langle \Phi_{K},X\rangle,\quad \forall X\in\mathfrak{k}, \end{align} $$

where $X_{M}(m) :=\frac {d}{dt}\vert _{t=0}e^{-tX}m$ is the vector field on M generated by $X\in \mathfrak {k}$ .

Definition 2.7. Let $\dim _{K}(M):=\min _{m\in M}\dim \mathfrak {k}_{m}$ . An element $\xi \in \mathfrak {k}^{*}$ is a weakly regular value of $\Phi _{K}$ if for all $m\in \Phi _{K}^{-1}(\xi )$ we have $\dim \mathfrak {k}_{m}=\dim _{K}(M)$ .

When $\xi \in \mathfrak {k}^{*}$ is a weakly regular value of $\Phi _{K}$ , the constant rank theorem tells us that $\Phi _{K}^{-1}(\xi )$ is a submanifold of M stable under the action of the stabilizer subgroup $K_{\xi }$ . We see then that the reduced space

(11) $$ \begin{align} M_{\xi}:=\Phi_{K}^{-1}(\xi)/K_{\xi} \end{align} $$

is a symplectic orbifold.

Let $T\subset K$ be a maximal torus with Lie algebra $\mathfrak {t}$ . We consider the lattice $\wedge :=\frac {1}{2\pi }\ker (\exp :\mathfrak {t}\to T)$ and the dual lattice $\wedge ^{*}\subset \mathfrak {t}^{*}$ defined by $\wedge ^{*}=\hom (\wedge ,\mathbb {Z})$ . We remark that $i\eta $ is a differential of a character of T if and only if $\eta \in \wedge ^{*}$ . The $\mathbb {Q}$ -vector space generated by the lattice $\wedge ^{*}$ is denoted by $\mathfrak {t}^{*}_{\mathbb {Q}}$ : The vectors belonging to $\mathfrak {t}^{*}_{\mathbb {Q}}$ are designed as rational. An affine subspace $V\subset \mathfrak {t}^{*}$ is called rational if it is affinely generated by its rational points.

We also fix a closed positive Weyl chamber $\mathfrak {t}^{*}_{\geq 0}$ . For each relatively open face $\sigma \subset \mathfrak {t}^{*}_{\geq 0}$ , the stabilizer $K_{\xi }$ of points $\xi \in \sigma $ under the coadjoint action does not depend on $\xi $ and will be denoted by $K_{\sigma }$ . The Lie algebra $\mathfrak {k}_{\sigma }$ decomposes into its semisimple and central parts $\mathfrak {k}_{\sigma }=[\mathfrak {k}_{\sigma },\mathfrak {k}_{\sigma }]\oplus \mathfrak {z}_{\sigma }$ . The subspace $\mathfrak {z}_{\sigma }^{*}$ is defined to be the annihilator of $[\mathfrak {k}_{\sigma },\mathfrak {k}_{\sigma }]$ or, equivalently, the fixed point set of the coadjoint $K_{\sigma }$ action. Notice that $\mathfrak {z}_{\sigma }^{*}$ is a rational subspace of $\mathfrak {t}^{*}$ and that the face $\sigma $ is an open cone of $\mathfrak {z}_{\sigma }^{*}$ ,

We suppose that the moment map $\Phi _{K}$ is proper. The convexity theorem [1, 10, 16, 35, 22] tells us that $\Delta _{K}(M):=\mathrm {Image}(\Phi _{K})\, \bigcap \, \mathfrak {t}_{\geq 0}^{*}$ is a closed, convex, locally polyhedral set.

Definition 2.8. We denote by $\Delta _{K}(M)^{0}$ the subset of $\Delta _{K}(M)$ formed by the weakly regular values of the moment map $\Phi _{K}$ contained in $\Delta _{K}(M)$ .

We will use the following remark in the next sections.

Lemma 2.9. The subset $\Delta _{K}(M)^{0}\cap \mathfrak {t}_{\mathbb {Q}}^{*}$ is dense in $\Delta _{K}(M)$ .

Proof. Let us first explain why $\Delta _{K}(M)^{0}$ is a dense open subset of $\Delta _{K}(M)$ . There exists a unique open face $\tau $ of the Weyl chamber $\mathfrak {t}^{*}_{\geq 0}$ such as $\Delta _{K}(M)\cap \tau $ is dense in $\Delta _{K}(M)$ : $\tau $ is called the principal face in [22]. The principal-cross-section theorem [22] tells us that $Y_{\tau } :=\Phi ^{-1}(\tau )$ is a symplectic $K_{\tau }$ -manifold, with a trivial action of $[K_{\tau },K_{\tau }]$ . The line bundle $\mathcal {L}_{\tau }:=\mathcal {L}\vert _{Y_{\tau }}$ prequantizes the symplectic structure on $Y_{\tau }$ , and relations (10) show that $[K_{\tau },K_{\tau }]$ acts trivially on $\mathcal {L}_{\tau }$ . Moreover, the restriction of $\Phi _{K}$ on $Y_{\tau }$ is the moment map $\Phi _{\tau }: Y_{\tau }\to \mathfrak {z}_{\tau }^{*}$ associated to the action of the torus $Z_{\tau }=\exp (\mathfrak {z}_{\tau })$ on $\mathcal {L}_{\tau }$ .

Let $I\subset \mathfrak {z}_{\tau }^{*}$ be the smallest affine subspace containing $\Delta _{K}(M)$ . Let $\mathfrak {z}_{I}\subset \mathfrak {z}_{\tau }$ be the annihilator of the subspace parallel to I: Relations (10) show that $\mathfrak {z}_{I}$ is the generic infinitesimal stabilizer of the $\mathfrak {z}_{\tau }$ -action on $Y_{\tau }$ . Hence, $\mathfrak {z}_{I}$ is the Lie algebra of the torus $Z_{I}:=\exp (\mathfrak {z}_{I})$ .

We see then that any regular value of $\Phi _{\tau }: Y_{\tau }\to I$ , viewed as a map with codomain I, is a weakly regular value of the moment map $\Phi _{K}$ . It explains why $\Delta _{K}(M)^{0}$ is a dense open subset of $\Delta _{K}(M)$ .

As the convex set $\Delta _{K}(M)\cap \tau $ is equal to $\Delta _{Z_{\tau }}(Y_{\tau }):=\mathrm {Image}(\Phi _{\tau })$ , it is sufficient to check that $\Delta _{Z_{\tau }}(Y_{\tau })^{0}\cap \mathfrak {t}_{\mathbb {Q}}^{*}$ is dense in $\Delta _{Z_{\tau }}(Y_{\tau })$ . The subtorus $Z_{I}\subset Z_{\tau }$ acts trivially on $Y_{\tau }$ , and it acts on the line bundle $\mathcal {L}_{\tau }$ through a character $\chi $ . Let $\eta \in \wedge ^{*}\cap \mathfrak {t}^{*}_{\tau }$ such that $d\chi =i\eta \vert _{\mathfrak {z}_{I}}$ . The affine subspace I which is equal to $\eta +(\mathfrak {z}_{I})^{\perp }$ is rational. Since the open subset $\Delta _{Z_{\tau }}(Y_{\tau })^{0}$ generates the rational affine subspace I, we can conclude that $\Delta _{Z_{\tau }}(Y_{\tau })^{0}\cap \mathfrak {t}_{\mathbb {Q}}^{*}$ is dense in $\Delta _{Z_{\tau }}(Y_{\tau })$ .

2.4 Weinstein’s theorem

Let $\tilde {a}\in \tilde {\mathcal {C}}_{\mathrm {hol}}$ . Consider the Hamiltonian action of the group G on the coadjoint orbit $\tilde {G}\tilde {a}$ . The moment map $\Phi _{G}^{\tilde {a}}:\tilde {G}\tilde {a}\to \mathfrak {g}^{*}$ corresponds to the restriction of the projection $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}$ to $\tilde {G}\tilde {a}$ . In this setting, the following conditions holds:

1. 1. The action of G on $\tilde {G}\tilde {a}$ is proper.

2. 2. The moment map $\Phi _{G}^{\tilde {a}}$ is a proper map since the map $\langle \Phi _{G}^{\tilde {a}},z\rangle $ is proper (see Lemma 2.3).

Conditions 1 and 2 impose that the image of $\Phi _{G}^{\tilde {a}}$ is contained in the open subset $\mathfrak {g}^{*}_{se}$ of strongly elliptic elements [31]. Thus, the G-orbits contained in the image of $\Phi _{G}^{\tilde {a}}$ are parametrized by the following subset of the holomorphic chamber $\mathcal {C}_{\mathrm {hol}}$ :

$$ \begin{align*}\Delta_{G}(\tilde{G}\tilde{a}):=\mathrm{Image}(\Phi_{G}^{\tilde{a}})\, \bigcap \, \mathfrak{t}^{*}_{\geq 0}. \end{align*} $$

We notice that $\Delta _{\mathrm {hol}}(\tilde {G},G)=\bigcup _{\tilde {a}\in \tilde {\mathcal {C}}_{\mathrm {hol}}}\{\tilde {a}\}\times \Delta _{G}(\tilde {G}\tilde {a})$ .

Like in Definition 2.7, an element $\xi \in \mathfrak {g}^{*}$ is a weakly regular value of $\Phi _{G}^{\tilde {a}}$ if for all $m\in (\Phi _{G}^{\tilde {a}})^{-1}(\xi )$ we have $\dim \mathfrak {g}_{m}=\min _{x\in \tilde {G}\tilde {a}}\dim (\mathfrak {g}_{x})$ . We denote by $\Delta _{G}(\tilde {G}\tilde {a})^{0}$ the set of elements $\xi \in \Delta _{G}(\tilde {G}\tilde {a})$ that are weakly regular for $\Phi _{G}^{\tilde {a}}$ .

Theorem 2.10 (Weinstein). For any $\tilde {a}\in \tilde {\mathcal {C}}_{\mathrm {hol}}$ , $\Delta _{G}(\tilde {G}\tilde {a})$ is a closed convex subset contained in $\mathcal {C}_{\mathrm {hol}}$ .

Proof. We recall briefly the arguments of the proof (see [38] or [31][§2]). Under Conditions 1 and 2, one checks easily that $Y_{\tilde {a}}:=(\Phi _{G}^{\tilde {a}})^{-1}(\mathfrak {k}^{*})$ is a smooth K-invariant symplectic submanifold of $\tilde {G}\tilde {a}$ such that

(12) $$ \begin{align} \tilde{G}\tilde{a}\simeq G\times_{K} Y_{\tilde{a}}. \end{align} $$

The moment map $\Phi _{K}^{\tilde {a}}:Y_{\tilde {a}}\to \mathfrak {k}^{*}$ , which corresponds to the restriction of the map $\Phi _{G}^{\tilde {a}}$ to $Y_{\tilde {a}}$ , is a proper map. Hence, the convexity theorem tells us that $\Delta _{K}(Y_{\tilde {a}}):=\mathrm {Image}(\Phi _{K}^{\tilde {a}})\bigcap \mathfrak {t}^{*}_{\geq 0}$ is a closed, convex, locally polyhedral set. Thanks to the isomorphism (12), we see that $\Delta _{G}(\tilde {G}\tilde {a})$ coincides with the closed convex subset $\Delta _{K}(Y_{\tilde {a}})$ . The proof is completed.

The next lemma is used in §4.1.

Lemma 2.11. Let $\tilde {a}\in \tilde {\mathcal {C}}_{\mathrm {hol}}$ be a rational element. Then $\Delta _{G}(\tilde {G}\tilde {a})^{0}\cap \mathfrak {t}_{\mathbb {Q}}^{*}$ is dense in $\Delta _{G}(\tilde {G}\tilde {a})$ .

Proof. Thanks to equation (12), we know that $\Delta _{G}(\tilde {G}\tilde {a})=\Delta _{K}(Y_{\tilde {a}})$ . Relation (12) shows also that $\Delta _{G}(\tilde {G}\tilde {a})^{0}=\Delta _{K}(Y_{\tilde {a}})^{0}$ . Let $N\geq 1$ such that $\tilde {\mu }=N\tilde {a}\in \wedge ^{*}\cap \mathcal {C}_{\mathrm {hol}}$ . The stabilizer subgroup $\tilde {G}_{\tilde {\mu }}$ is compact, equal to $\tilde {K}_{\tilde {\mu }}$ . Let us denote by $\mathbb {C}_{\tilde {\mu }}$ the one-dimensional representation of $\tilde {K}_{\tilde {\mu }}$ associated to $\tilde {\mu }$ . The convex set $\Delta _{G}(\tilde {G}\tilde {a})$ is equal to $\frac {1}{N}\Delta _{G}(\tilde {G}\tilde {\mu })$ , so it is sufficient to check that $\Delta _{G}(\tilde {G}\tilde {\mu })^{0}\cap \mathfrak {t}_{\mathbb {Q}}^{*}=\Delta _{K}(Y_{\tilde {\mu }})^{0}\cap \mathfrak {t}_{\mathbb {Q}}^{*}$ is dense in $\Delta _{G}(\tilde {G}\tilde {\mu })=\Delta _{K}(Y_{\tilde {\mu }})$ . The coadjoint orbit $\tilde {G}\tilde {\mu }$ is prequantized by the line bundle $\tilde {G}\times _{K_{\tilde {\mu }}}\mathbb {C}_{\tilde {\mu }}$ , and the symplectic slice $Y_{\tilde {\mu }}$ is prequantized by the line bundle $\mathcal {L}_{\tilde {\mu }}:=\tilde {G}\times _{K_{\tilde {\mu }}}\mathbb {C}_{\tilde {\mu }}\vert _{Y_{\tilde {\mu }}}$ . Thanks to Lemma 2.9, we know that $\Delta _{K}(Y_{\tilde {\mu }})^{0}\cap \mathfrak {t}_{\mathbb {Q}}^{*}$ is dense in $\Delta _{K}(Y_{\tilde {\mu }})$ : The proof is complete.

3 Holomorphic discrete series

3.1 Definition

We return to the framework of §2.1. We recall the notion of holomorphic discrete series representations associated to a Hermitian symmetric spaces $G/K$ . Let us introduce

$$ \begin{align*}\mathcal{C}_{\mathrm{hol}}^{\rho}:=\left\{ \xi\in\mathfrak{t}^{*}_{\geq 0} \vert\ (\xi,\beta)\geq (2\rho_{n},\beta) , \ \forall \beta\in \mathfrak{R}_{n}^{+,z}\right\}, \end{align*} $$

where $2\rho _{n}=\sum _{\beta \in \mathfrak {R}_{n}^{+,z}}\beta $ is W-invariant.

Lemma 3.1.

1. 1. We have $\mathcal {C}_{\mathrm {hol}}^{\rho }\subset \mathcal {C}_{\mathrm {hol}}$ .

2. 2. For any $\xi \in \mathcal {C}_{\mathrm {hol}}$ , there exists $N\geq 1$ such that $N\xi \in \mathcal {C}_{\mathrm {hol}}^{\rho }$ .

Proof. The first point is due to the fact that $(\beta _{0},\beta _{1})\geq 0$ for any $\beta _{0},\beta _{1}\in \mathfrak {R}_{n}^{+,z}$ . The second point is obvious.

We will be interested in the following subset of dominant weights:

$$ \begin{align*}\widehat{G}_{\mathrm{hol}}:=\wedge^{*}_{+}\bigcap \mathcal{C}_{\mathrm{hol}}^{\rho}. \end{align*} $$

Let $\mathrm {Sym}(\mathfrak {p})$ be the symmetric algebra of the complex K-module $(\mathfrak {p},\mathrm {ad}(z))$ .

Theorem 3.2 (Harish–Chandra). For any $\lambda \in \widehat {G}_{\mathrm {hol}}$ , there exists an irreducible unitary representation of G, denoted by $V^{G}_{\lambda }$ , such that the vector space of K-finite vectors is $V^{G}_{\lambda }\vert _{K}:=V_{\lambda }^{K}\otimes \mathrm {Sym}(\mathfrak {p})$ .

The set $V^{G}_{\lambda },\lambda \in \widehat {G}_{\mathrm {hol}}$ corresponds to the holomorphic discrete series representations associated to the complex structure $\mathrm {ad}(z)$ .

3.2 Restriction

We come back to the framework of §2.2. We consider two compatible Hermitian symmetric spaces $G/K\subset \tilde {G}/\tilde {K}$ , and we look after the restriction of holomorphic discrete series representations of $\tilde {G}$ to the subgroup G.

Let $\tilde {\lambda }\in \widehat {\tilde {G}}_{\mathrm {hol}}$ . Since the representation $V^{\tilde {G}}_{\tilde {\lambda }}$ is discretely admissible relatively to the circle group $\exp (\mathbb {R} z)$ , it is also discretely admissible relatively to G. We can be more precise [15, 24, 21]:

Proposition 3.3. We have an Hilbertian direct sum

$$ \begin{align*}V^{\tilde{G}}_{\tilde{\lambda}}\vert_{G}=\bigoplus_{\lambda\in \widehat{G}_{\mathrm{hol}}} m_{\tilde{\lambda}}^{\lambda} \ V^{G}_{\lambda}, \end{align*} $$

where the multiplicity $m_{\tilde {\lambda }}^{\lambda } :=[V^{G}_{\lambda }:V^{\tilde {G}}_{\tilde {\lambda }}]$ is finite for any $\lambda $ .

The Hermitian $\tilde {K}$ -vector space $\tilde {\mathfrak {p}}$ , when restricted to the K-action, admits an orthogonal decomposition $\tilde {\mathfrak {p}}=\mathfrak {p}\oplus \mathfrak {q}$ . Notice that the symmetric algebra $\mathrm {Sym}(\mathfrak {q})$ is an admissible K-module.

In [15], H. P. Jakobsen and M. Vergne obtained the following nice characterization of the multiplicities $[V^{G}_{\lambda }:V^{\tilde {G}}_{\tilde {\lambda }}]$ . Another proof is given in [31], §4.4.

Theorem 3.4 (Jakobsen–Vergne). Let $(\tilde {\lambda },\lambda )\in \widehat {\tilde {G}}_{\mathrm {hol}}\times \widehat {G}_{\mathrm {hol}}$ . The multiplicity $[V^{G}_{\lambda }:V^{\tilde {G}}_{\tilde {\lambda }}]$ is equal to the multiplicity of the representation $V^{K}_{\lambda }$ in $\mathrm {Sym}(\mathfrak {q})\otimes V^{\tilde {K}}_{\tilde {\lambda }}\vert _{K} $ .

3.3 Discrete analogues of $\Delta _{\mathrm {hol}}(\tilde {G},G)$

We define the following discrete analogues of the cone $\Delta _{\mathrm {hol}}(\tilde {G},G)$ :

(13) $$ \begin{align} \Pi_{\mathrm{hol}}^{\mathbb{Z}}(\tilde{G},G):=\left\{(\tilde{\lambda},\lambda)\in \widehat{\tilde{G}}_{\mathrm{hol}}\times\widehat{G}_{\mathrm{hol}}\; \ [V^{G}_{\lambda}:V^{\tilde{G}}_{\tilde{\lambda}}]\neq 0 \right\}, \end{align} $$

and

(14) $$ \begin{align} \Pi_{\mathrm{hol}}^{\mathbb{Q}}(\tilde{G},G):=\left\{(\tilde{\xi},\xi)\in \tilde{\mathcal{C}}_{\mathrm{hol}}\times\mathcal{C}_{\mathrm{hol}}\; \ \exists N\geq 1,\ (N\xi,N\tilde{\xi})\in \Pi_{\mathrm{hol}}^{\mathbb{Z}}(\tilde{G},G)\right\}. \end{align} $$

We have the following key fact.

Proposition 3.5. ${}$

• • $\Pi _{\mathrm {hol}}^{\mathbb {Z}}(\tilde {G},G)$ is a subset of $ \tilde {\wedge }^{*} \times \wedge ^{*}$ stable under the addition.

• • $\Pi _{\mathrm {hol}}^{\mathbb {Q}}(\tilde {G},G)$ is a $\mathbb {Q}$ -convex cone of the $\mathbb {Q}$ -vector space $\tilde {\mathfrak {t}}^{*}_{\mathbb {Q}} \times \mathfrak {t}^{*}_{\mathbb {Q}}$ .

Proof. Suppose that $a_{1}:=(\tilde {\lambda }_{1},\lambda _{1})$ and $a_{2}:=(\tilde {\lambda }_{2}, \lambda _{2})$ belongs to $\Pi _{\mathrm {hol}}^{\mathbb {Z}}(\tilde {G},G)$ . Thanks to Theorem 3.4, we know that the K-modules $\mathrm {Sym}(\mathfrak {q})\otimes (V^{K}_{\lambda _{j}})^{*}\otimes V^{\tilde {K}}_{\tilde {\lambda }_{j}}\vert _{K}$ possess a nonzero invariant vector $\phi _{j}$ , for $j=1,2$ .

Let $\mathbb {X}:=\overline {K/T}\times \tilde {K}/\tilde {T}$ be the product of flag manifolds. The complex structure is normalized so that $\textbf { T}_{([e],[\tilde {e}])}\mathbb {X}\simeq \mathfrak {n}_{-}\oplus \tilde {\mathfrak {n}}_{+}$ , where $\mathfrak {n}_{-}=\sum _{\alpha <0}(\mathfrak {k}_{\mathbb {C}})_{\alpha }$ and $\tilde {\mathfrak {n}}_{+}=\sum _{\tilde {\alpha }>0}(\tilde {\mathfrak {k}}_{\mathbb {C}})_{\tilde {\alpha }}$ . We associate to each data $a_{j}$ , the holomorphic line bundle $\mathcal {L}_{j}:=K\times _{T} \mathbb {C}_{-\lambda _{j}}\boxtimes \tilde {K}\times _{\tilde {T}} \mathbb {C}_{-\tilde {\lambda }_{j}}$ on $\mathbb {X}$ . Let $H^{0}(\mathbb {X},\mathcal {L}_{j})$ be the space of holomorphic sections of the line bundle $\mathcal {L}_{j}$ . The Borel–Weil theorem tells us that $H^{0}(\mathbb {X},\mathcal {L}_{j})\simeq (V^{K}_{\lambda _{j}})^{*}\otimes V^{\tilde {K}}_{\tilde {\lambda }_{j}}\vert _{K}$ , $\forall j\in \{1,2\}$ .

We have $\phi _{j}\in \left [\mathrm {Sym}(\mathfrak {q})\otimes H^{0}(\mathbb {X},\mathcal {L}_{j})\right ]^{K}$ , $\forall j$ , and then $\phi _{1}\phi _{2}\in \mathrm {Sym}(\mathfrak {q})\otimes H^{0}(\mathbb {X},\mathcal {L}_{1}\otimes \mathcal {L}_{2})$ is a nonzero invariant vector. Hence, $[\mathrm {Sym}(\mathfrak {q})\otimes (V^{K}_{\lambda _{1}+\lambda _{2}})^{*}\otimes V^{\tilde {K}}_{\tilde {\lambda }_{1}+\tilde {\lambda }_{2}}\vert _{K}]^{K}\neq 0$ . Thanks to Theorem 3.4, we can conclude that $a_{1}+a_{2}=(\tilde {\lambda }_{1}+\tilde {\lambda }_{2},\lambda _{1}+\lambda _{2})$ belongs to $\Pi _{\mathrm {hol}}^{\mathbb {Z}}(\tilde {G},G)$ . The first point is proved. From the first point, one checks easily that

1. - $\Pi _{\mathrm {hol}}^{\mathbb {Q}}(\tilde {G},G)$ is stable under addition,

2. - $\Pi _{\mathrm {hol}}^{\mathbb {Q}}(\tilde {G},G)$ is stable by expansion by a nonnegative rational number.

The second point is settled.

3.4 Riemann–Roch numbers

We come back to the framework of §2.3.

We associate to a dominant weight $\mu \in \wedge ^{*}_{+}$ the (possibly singular) symplectic reduced space $M_{\mu }:=\Phi _{K}^{-1}(\mu )/K_{\mu }$ and the (possibly singular) line bundle over $M_{\mu }$ :

$$ \begin{align*}\mathcal{L}_{\mu}:=\left(\mathcal{L}\vert_{\Phi_{K}^{-1}(\mu)}\otimes \mathbb{C}_{-\mu}\right)/K_{\mu}. \end{align*} $$

$\underline {\mbox {Suppose first that }\mu \mbox { is a weakly regular value of }\Phi _{K}.}$ Then $M_{\mu }$ is an orbifold equipped with a symplectic structure $\Omega _{\mu }$ , and $\mathcal {L}_{\mu }$ is a line orbi-bundle over $M_{\mu }$ that prequantizes the symplectic structure. By choosing an almost complex structure on $M_{\mu }$ compatible with $\Omega _{\mu }$ , we get a decomposition $\wedge \textbf { T}^{*} M_{\mu }\otimes \mathbb {C} = \oplus _{i,j}\wedge ^{i,j} \textbf { T}^{*} M_{\mu }$ of the bundle of differential forms. Using Hermitian structures in the tangent bundle $\textbf { T} M_{\mu }$ of $M_{\mu }$ and in the fibers of $\mathcal {L}_{\mu }$ , we define a Dolbeaut–Dirac operator

$$ \begin{align*}D^{+}_{\mu} : \mathcal{A}^{0,+}(M_{\mu},\mathcal{L}_{\mu})\longrightarrow \mathcal{A}^{0,-}(M_{\mu},\mathcal{L}_{\mu}), \end{align*} $$

where $\mathcal {A}^{i,j}(M_{\mu },\mathcal {L}_{\mu })=\Gamma (M_{\mu },\wedge ^{i,j} \textbf { T}^{*} M_{\mu }\otimes \mathcal {L}_{\mu })$ .

Definition 3.6. Let $\mu \in \wedge ^{*}_{+}$ be a weakly regular value of the moment map $\Phi _{K}$ . The Riemann–Roch number $RR(M_{\mu },\mathcal {L}_{\mu })\in \mathbb {Z}$ is defined as the index of the elliptic operator $D^{+}_{\mu }$ : $RR(M_{\mu },\mathcal {L}_{\mu })= \dim (ker(D^{+}_{\mu })) -\dim (coker(D^{+}_{\mu }))$ .

$\underline {\mbox {Suppose that }\mu \notin \Delta _{K}(M).}$ Then $M_{\mu }=\emptyset $ , and we take $RR(M_{\mu },\mathcal {L}_{\mu })=0$ .

$\underline {\mbox {Suppose now that }\mu \in \Delta _{K}(M)\mbox { is not (necessarily) a weakly regular value of }\Phi _{K}.}$ Take a small element $\epsilon \in \mathfrak {t}^{*}$ such that $\mu +\epsilon $ is a weakly regular value of $\Phi _{K}$ belonging to $\Delta _{K}(M)$ . We consider the symplectic orbifold $M_{\mu +\epsilon }$ : If $\epsilon $ is small enough,

$$ \begin{align*}\mathcal{L}_{\mu,\epsilon}:=\left(\mathcal{L}\vert_{\Phi_{K}^{-1}(\mu+\epsilon)}\otimes \mathbb{C}_{-\mu}\right)/K_{\mu+\epsilon}. \end{align*} $$

is a line orbi-bundle over $M_{\mu +\epsilon }$ .

We have the following important result (see §3.4.3 in [34]).

Proposition 3.7. Let $\mu \in \Delta _{K}(M)\cap \wedge ^{*}$ . The Riemann–Roch number $RR(M_{\mu +\epsilon },\mathcal {L}_{\mu ,\epsilon })$ do not depend on the choice of $\epsilon $ small enough so that $\mu +\epsilon \in \Delta _{K}(M)$ is a weakly regular value of $\Phi _{K}$ .

We can now introduce the following definition.

Definition 3.8. Let $\mu \in \wedge ^{*}_{+}$ . We define

$$ \begin{align*}\mathcal{Q}(M_{\mu},\Omega_{\mu})= \begin{cases} 0 \hspace{26.5mm} \mathrm{if }\quad \mu\notin \Delta_{K}(M),\\ RR(M_{\mu+\epsilon},\mathcal{L}_{\mu,\epsilon})\hspace{4mm} \mathrm{if }\quad \mu\in \Delta_{K}(M). \end{cases} \end{align*} $$

Above, $\epsilon $ is chosen small enough so that $\mu +\epsilon \in \Delta _{K}(M)$ is a weakly regular value of $\Phi _{K}$ .

Let $n\geq 1$ . The manifold M, equipped with the symplectic structure $n\Omega $ , is prequantized by the line bundle $\mathcal {L}^{\otimes n}$ : The corresponding moment map is $n\Phi _{K}$ . For any dominant weight $\mu \in \wedge ^{*}_{+}$ , the symplectic reduction of $(M,n\Omega )$ relatively to the weight $n\mu $ is $(M_{\mu },n\Omega _{\mu })$ . Like in Definition 3.8, we consider the following Riemann–Roch numbers

$$ \begin{align*}\mathcal{Q}(M_{\mu},n\Omega_{\mu})= \begin{cases} 0 \hspace{33.4mm} \mathrm{if }\quad \mu\notin \Delta_{K}(M),\\ RR(M_{\mu+\epsilon},(\mathcal{L}_{\mu,\epsilon})^{\otimes n})\hspace{4mm} \mathrm{if }\quad \mu\in \Delta_{K}(M)\ \mathrm{and} \ \|\epsilon\|<<1. \end{cases} \end{align*} $$

The Kawasaki–Riemann–Roch formula shows that $n\geq 1\mapsto \mathcal {Q}(M_{\mu },n\Omega _{\mu })$ is a quasi-polynomial map [37, 23]. When $\mu $ is a weakly regular value of $\Phi _{K}$ , we denote by $\mathrm {vol}(M_{\mu }):=\frac {1}{d_{\mu }}\int _{M_{\mu }}(\frac {\Omega _{\mu }}{2\pi })^{\frac {\dim M_{\mu }}{2}}$ the symplectic volume of the symplectic orbifold $(M_{\mu },\Omega _{\mu })$ . Here, $d_{\mu }$ is the generic value of the map $m\in \Phi _{K}^{-1}(\mu )\mapsto \mathrm {cardinal}(K_{m}/K^{0}_{m})$ .

The following proposition is a direct consequence of the Kawasaki–Riemann–Roch formula (see [23] or §1.3.4 in [30]).

Proposition 3.9. Let $\mu \in \Delta _{K}(M)\cap \wedge ^{*}_{+}$ be a weakly regular value of $\Phi _{K}$ . Then we have $\mathcal {Q}(M_{\mu },n\Omega _{\mu })\ \sim \ \mathrm {vol}(M_{\mu }) \,n^{\frac {\dim M_{\mu }}{2}}$ when $n\to \infty $ . In particular, the map $n\geq 1\mapsto \mathcal {Q}(M_{\mu },n\Omega _{\mu })$ is nonzero.

3.5 Quantization commutes with reduction

Let us explain the “quantization commutes with reduction” theorem proved in [31].

We fix $\tilde {\lambda }\in \widehat {\tilde {G}}_{\mathrm {hol}}$ . The coadjoint orbit $\tilde {G}\tilde {\lambda }$ is prequantized by the line bundle $\tilde {G}\times _{K_{\tilde {\lambda }}}\mathbb {C}_{\tilde {\lambda }}$ , and the moment map $\Phi _{G}^{\tilde {\lambda }}:\tilde {G}\tilde {\lambda }\to \mathfrak {g}^{*}$ corresponding to the G-action on $\tilde {G}\times _{K_{\tilde {\lambda }}}\mathbb {C}_{\tilde {\lambda }}$ is equal to the restriction of the map $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}$ to $\tilde {G}\tilde {\lambda }$ .

The symplectic slice $Y_{\tilde {\lambda }}=(\Phi _{G}^{\tilde {\lambda }})^{-1}(\mathfrak {k}^{*})$ is prequantized by the line bundle $\mathcal {L}_{\tilde {\lambda }}:=\tilde {G}\times _{K_{\tilde {\lambda }}}\mathbb {C}_{\tilde {\lambda }}\vert _{Y_{\tilde {\lambda }}}$ . The moment map $\Phi _{K}^{\tilde {\lambda }}:Y_{\tilde {\lambda }}\to \mathfrak {k}^{*}$ corresponding to the K-action is equal to the restriction of $\Phi _{G}^{\tilde {\lambda }}$ to $Y_{\tilde {\lambda }}$ .

For any $\lambda \in \widehat {G}_{\mathrm {hol}}$ , we consider the (possibly singular) symplectic reduced space

$$ \begin{align*}\mathbb{X}_{\tilde{\lambda},\lambda}:=(\Phi_{K}^{\tilde{\lambda}})^{-1}(\lambda)/K_{\lambda}, \end{align*} $$

equipped with the reduced symplectic form $\Omega _{\tilde {\lambda },\lambda }$ , and the (possibly singular) line bundle

$$ \begin{align*}\mathbb{L}_{\tilde{\lambda},\lambda}:=\left(\mathcal{L}_{\tilde{\lambda}}\vert_{(\Phi_{K}^{\tilde{\lambda}})^{-1}(\lambda)}\otimes \mathbb{C}_{-\lambda}\right)/K_{\lambda}. \end{align*} $$

Thanks to Definition 3.8, the geometric quantization $\mathcal {Q}(\mathbb {X}_{\tilde {\lambda },\lambda },\Omega _{\tilde {\lambda },\lambda })\in \mathbb {Z}$ of those compact symplectic spaces $(\mathbb {X}_{\tilde {\lambda },\lambda },\Omega _{\tilde {\lambda },\lambda })$ are well-defined even if they are singular. In particular, $\mathcal {Q}(\mathbb {X}_{\tilde {\lambda },\lambda },\Omega _{\tilde {\lambda },\lambda })=0$ when $\mathbb {X}_{\tilde {\lambda },\lambda }=\emptyset $ .

The following theorem is proved in [31].

Theorem 3.10. Let $\tilde {\lambda }\in \widehat {\tilde {G}}_{\mathrm {hol}}$ . We have an Hilbertian direct sum

$$ \begin{align*}V^{\tilde{G}}_{\tilde{\lambda}}\vert_{G}=\bigoplus_{\lambda\in \widehat{G}_{\mathrm{hol}}} \mathcal{Q}(\mathbb{X}_{\tilde{\lambda},\lambda},\Omega_{\tilde{\lambda},\lambda}) \ V^{G}_{\lambda}. \end{align*} $$

It means that, for any $\lambda \in \widehat {G}_{\mathrm {hol}}$ , the multiplicity of the representation $V^{G}_{\lambda }$ in the restriction $V^{\tilde {G}}_{\tilde {\lambda }}\vert _{G}$ is equal to the geometric quantization $\mathcal {Q}(\mathbb {X}_{\tilde {\lambda },\lambda },\Omega _{\tilde {\lambda },\lambda })$ of the (compact) reduced space $\mathbb {X}_{\tilde {\lambda },\lambda }$ .

Remark 3.11. Let $({\tilde {\lambda },\lambda })\in \widehat {\tilde {G}}_{\mathrm {hol}}\times \widehat {G}_{\mathrm {hol}}$ . Theorem 3.10. shows that

$$ \begin{align*}\left[V^{G}_{n\lambda} : V^{\tilde{G}}_{n\tilde{\lambda}}\right]= \mathcal{Q}(\mathbb{X}_{\tilde{\lambda},\lambda},n\Omega_{\tilde{\lambda},\lambda}) \end{align*} $$

for any $n\geq 1$ .

4 Proofs of the main results

We come back to the setting of §2.2: $G/K$ is a complex submanifold of a Hermitian symmetric space $\tilde {G}/\tilde {K}$ . It means that there exits a $\tilde {K}$ -invariant element $z\in \mathfrak {k}$ such that $\mathrm {ad}(z)$ defines complex structures on $\tilde {\mathfrak {p}}$ and $\mathfrak {p}$ . We consider the orthogonal decomposition $\tilde {\mathfrak {p}}=\mathfrak {p}\oplus \mathfrak {q}$ , and we denote by $\mathrm {Sym}(\mathfrak {q})$ the symmetric algebra of the complex K-module $(\mathfrak {q},\mathrm {ad}(z))$ .

4.1 Proof of Theorem A

The set $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is equal to $\bigcup _{\tilde {a}\in \tilde {\mathcal {C}}_{\mathrm {hol}}}\{\tilde {a}\}\times \Delta _{G}(\tilde {G}\tilde {a})$ . We define

$$ \begin{align*}\Delta_{\mathrm{hol}}(\tilde{G},G)^{0}:=\bigcup_{\tilde{a}\in \tilde{\mathcal{C}}_{\mathrm{hol}}} \{\tilde{a}\} \times \Delta_{G}(\tilde{G}\tilde{a})^{0}. \end{align*} $$

We start with the following result.

Lemma 4.1. The set $\Delta _{\mathrm {hol}}(\tilde {G},G)^{0}\bigcap \tilde {\mathfrak {t}}^{*}_{\mathbb {Q}} \times \mathfrak {t}^{*}_{\mathbb {Q}}$ is dense in $\Delta _{\mathrm {hol}}(\tilde {G},G)$ .

Proof. Let $(\tilde {\xi },\xi )\in \Delta _{\mathrm {hol}}(\tilde {G},G)$ : take $\tilde {g}\in \tilde {G}$ such that $\xi =\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}(\tilde {g}\tilde {\xi })$ . We consider a sequence $\tilde {\xi }_{n}\in \tilde {\mathcal {C}}_{\mathrm {hol}}\cap \tilde {\mathfrak {t}}^{*}_{\mathbb {Q}}$ converging to $\tilde {\xi }$ . Then $\xi _{n}:=\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}(\tilde {g}\tilde {\xi }_{n})$ is a sequence of $\mathcal {C}^{0}_{G/K}$ converging to $\xi \in \mathcal {C}_{\mathrm {hol}}$ . Since the map $\mathrm {p}: \mathcal {C}_{G/K}^{0}\to \mathcal {C}_{\mathrm {hol}}$ is continuous (see Lemma 2.4), the sequence $\eta _{n}:=\mathrm{p}(\xi _{n})$ converges to $\mathrm {p}(\xi )=\xi $ . By definition, we have $\eta _{n}\in \Delta _{G}(\tilde {G}\,\tilde {\xi }_{n})$ for any $n\in \mathbb {N}$ . Since $\tilde {\xi }_{n}$ are rational, each subset $\Delta _{G}(\tilde {G}\tilde {\xi }_{n})^{0}\cap \mathfrak {t}_{\mathbb {Q}}^{*}$ is dense in $\Delta _{G}(\tilde {G}\tilde {\xi }_{n})$ (see Lemma 2.11). Hence, $\forall n\in \mathbb {N}$ , there exists $\zeta _{n}\in \Delta _{G}(\tilde {G}\,\tilde {\xi }_{n})^{0}\cap \mathfrak {t}^{*}_{\mathbb {Q}}$ such that $\|\zeta _{n}-\eta _{n}\|\leq 2^{-n}$ . Finally, we see that $(\tilde {\xi }_{n},\zeta _{n})$ is a sequence of rational elements of $\Delta _{\mathrm {hol}}(\tilde {G},G)^{0}$ converging to $(\xi ,\tilde {\xi })$ .

The main purpose of this section is the proof of the following theorem.

Theorem 4.2. For any rational element $(\tilde {\mu },\mu )$ of the holomorphic chamber $\tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ , the following statements hold:

• • If $\mu \in \Delta _{G}(\tilde {G}\tilde {\mu })^{0}$ , then $(\tilde {\mu },\mu )\in \Pi ^{\mathbb {Q}}_{\mathrm {hol}}(\tilde {G},G)$ .

• • If $(\tilde {\mu },\mu )\in \Pi ^{\mathbb {Q}}_{\mathrm {hol}}(\tilde {G},G)$ , then $\mu \in \Delta _{G}(\tilde {G}\tilde {\mu })$ .

In other words, we have the following inclusions:

$$ \begin{align*} \Delta_{\mathrm{hol}}(\tilde{G},G)^{0}\, \bigcap\, \tilde{\mathfrak{t}}^{*}_{\mathbb{Q}} \times \mathfrak{t}^{*}_{\mathbb{Q}} \quad \underset{(1)}{\Large{\subset}}\quad \Pi_{\mathrm{hol}}^{\mathbb{Q}}(\tilde{G},G) \quad \underset{(2)}{\Large{\subset}}\quad \Delta_{\mathrm{hol}}(\tilde{G},G). \end{align*} $$

Lemma 4.1 and Theorem 4.2 gives the important corollary.

Corollary 4.3. $\Pi _{\mathrm {hol}}^{\mathbb {Q}}(\tilde {G},G)$ is dense in $\Delta _{\mathrm {hol}}(\tilde {G},G)$ .

Proof of Theorem 4.2

Let $(\tilde {\mu },\mu )\in \Pi _{\mathrm {hol}}^{\mathbb {Q}}(\tilde {G},G)$ : There exists $N\geq 1$ such that $(N\tilde {\mu },N\mu )\in \Pi _{\mathrm {hol}}^{\mathbb {Z}}(\tilde {G},G)$ . The multiplicity $[V^{G}_{N\mu }:V^{\tilde {G}}_{N\tilde {\mu }}]$ is nonzero, and thanks to Theorem 3.10, it implies that the reduced space $\mathbb {X}_{N\tilde {\mu },N\mu }$ is nonempty. In other words, $(N\tilde {\mu },N\mu )\in \Delta _{\mathrm {hol}}(\tilde {G},G)$ . The inclusion $(2)$ is proven.

Let $(\tilde {\mu },\mu )\in \Delta _{\mathrm {hol}}(\tilde {G},G)^{0}\bigcap \mathfrak {t}^{*}_{\mathbb {Q}}\times \tilde {\mathfrak {t}}^{*}_{\mathbb {Q}}$ . There exists $N_{o}\geq 1$ such that $\lambda :=N_{o}\mu \in \widehat {G}_{\mathrm {hol}}$ , $\tilde {\lambda }:=N_{o}\tilde {\mu }\in \widehat {\tilde {G}}_{\mathrm {hol}}$ and $\lambda \in \Delta _{G}(\tilde {G}\tilde {\lambda })^{0}$ : The element $\lambda $ is a weakly regular value of the moment map $\tilde {G}\tilde {\lambda }\to \mathfrak {g}^{*}$ . Theorem 3.10 tells us that, for any $n\geq 1$ , the multiplicity $[V^{G}_{n\lambda }:V^{\tilde {G}}_{n\tilde {\lambda }}]$ is equal to Riemann–Roch number $\mathcal {Q}(\mathbb {X}_{\tilde {\lambda },\lambda }, n\Omega _{\tilde {\lambda },\lambda })$ . Since the map $n\mapsto \mathcal {Q}(\mathbb {X}_{\tilde {\lambda },\lambda }, n\Omega _{\tilde {\lambda },\lambda })$ is nonzero (see Proposition 3.9), we can conclude that there exists $n_{o}\geq 1$ such that $[V^{G}_{n_{o}\lambda }:V^{\tilde {G}}_{n_{o}\tilde {\lambda }}]\neq 0$ . In other words, we obtain $n_{o}N_{o}(\tilde {\mu },\mu )\in \Pi _{\mathrm {hol}}^{\mathbb {Z}}(\tilde {G},G)$ and so $(\tilde {\mu },\mu )\in \Pi _{\mathrm {hol}}^{\mathbb {Q}}(\tilde {G},G)$ . The inclusion $(1)$ is settled.

Now we can terminate the proof of Theorem A.

Thanks to Proposition 3.5, we know that $\Pi _{\mathrm {hol}}^{\mathbb {Q}}(\tilde {G},G)$ is a $\mathbb {Q}$ -convex cone. Since $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is a closed subset of $\tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ (see Proposition 2.6), we can conclude, by a density argument, that $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is a closed convex cone of $\tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ .

4.2 The affine variety $\tilde {K}_{\mathbb {C}}\times \mathfrak {q}$

Let $\tilde {\kappa }$ be the Killing form on the Lie algebra $\tilde {\mathfrak {g}}$ . We consider the $\tilde {K}$ -invariant symplectic structures $\Omega _{\tilde {\mathfrak {p}}}$ on $\tilde {\mathfrak {p}}$ , defined by the relation

$$ \begin{align*}\Omega_{\tilde{\mathfrak{p}}}(\tilde{Y},\tilde{Y}^{\prime})=\tilde{\kappa}(z,[\tilde{Y},\tilde{Y}^{\prime}]),\quad \forall \tilde{Y},\tilde{Y}^{\prime}\in\tilde{\mathfrak{p}}. \end{align*} $$

We notice that the complex structure $\mathrm {ad}(z)$ is adapted to $\Omega _{\tilde {\mathfrak {p}}}$ : $\Omega _{\tilde {\mathfrak {p}}}(\tilde {Y},\mathrm {ad}(z)\tilde {Y})>0$ if $\tilde {Y}\neq 0$ .

We denote by $\Omega _{\mathfrak {q}}$ the restriction of $\Omega _{\tilde {\mathfrak {p}}}$ on the symplectic subspace $\mathfrak {q}$ . The moment map $\Phi _{\mathfrak {q}}$ associated to the K-action on $(\mathfrak {q}, \Omega _{\mathfrak {q}})$ is defined by the relations $\langle \Phi _{\mathfrak {q}}(Y),X\rangle = \frac {-1}{2}\tilde {\kappa }([X,Y],[z,Y])$ , $\forall (X,Y) \in \mathfrak {p}\times \mathfrak {q}$ . In particular, $\langle \Phi _{\mathfrak {q}}(Y),z\rangle = \frac {-1}{2}\|Y\|^{2}$ , $\forall Y \in \mathfrak {q}$ , so the map $\langle \Phi _{\mathfrak {q}},z\rangle $ is proper.

The complex reductive group $\tilde {K}_{\mathbb {C}}$ is equipped with the following action of $\tilde {K}\times K$ : $(\tilde {k},k)\cdot a= \tilde {k} a k^{-1}$ . It has a canonical structure of a smooth affine variety. There is a diffeomorphism of the cotangent bundle $\textbf { T}^{*}\tilde {K}$ with $\tilde {K}_{\mathbb {C}}$ defined as follows. We identify $\textbf { T}^{*}\tilde {K}$ with $\tilde {K}\times \tilde {\mathfrak {k}}^{*}$ by means of left-translation and then with $\tilde {K}\times \tilde {\mathfrak {k}}$ by means of an invariant inner product on $\tilde {\mathfrak {k}}$ . The map $\varphi :\tilde {K}\times \tilde {\mathfrak {k}}\to \tilde {K}_{\mathbb {C}}$ given by $\varphi (a,X)=a e^{iX}$ is a diffeomorphism. If we use $\varphi $ to transport the complex structure of $\tilde {K}_{\mathbb {C}}$ to $\textbf { T}^{*}\tilde {K}$ , then the resulting complex structure on $\textbf { T}^{*}\tilde {K}$ is compatible with the symplectic structure on $\textbf { T}^{*}\tilde {K}$ so that $\textbf { T}^{*}\tilde {K}$ becomes a Kähler Hamiltonian $\tilde {K}\times K$ -manifold (see [11], §3). The moment map relative to the $\tilde {K}\times K$ -action is the proper map $\Phi _{\tilde {K}} \oplus \Phi _{K} : \textbf { T}^{*}\tilde {K} \to \tilde {\mathfrak {k}}^{*}\oplus \mathfrak {k}^{*}$ defined by $\Phi _{\tilde {K}}(\tilde {a},\tilde {\eta })=-\tilde {a}\tilde {\eta }$ and $\Phi _{K}(\tilde {a},\tilde {\eta })=\pi _{\mathfrak {k},\tilde {\mathfrak {k}}}(\tilde {\eta })$ . Here $\pi _{\mathfrak {k},\tilde {\mathfrak {k}}}: \tilde {\mathfrak {k}}^{*} \to \mathfrak {k}^{*}$ is the projection dual to the inclusion $\mathfrak {k} \hookrightarrow \tilde {\mathfrak {k}}$ of Lie algebras.

Finally, we consider the Kähler Hamiltonian $\tilde {K}\times K$ -manifold $\textbf { T}^{*}\tilde {K}\times \mathfrak {q}$ , where $\mathfrak {q}$ is equipped with the symplectic structure $\Omega _{\mathfrak {q}}$ . Let us denote by $\Phi : \textbf { T}^{*}\tilde {K}\times \mathfrak {q}\to \tilde {\mathfrak {k}}^{*}\oplus \mathfrak {k}^{*}$ the moment map relative to the $\tilde {K}\times K$ -action:

(15) $$ \begin{align} \Phi(\tilde{a},\tilde{\eta},Y)=\left(-\tilde{a}\tilde{\eta},\pi_{\mathfrak{k},\tilde{\mathfrak{k}}}(\tilde{\eta})+\Phi_{\mathfrak{q}}(Y)\right). \end{align} $$

Since $\Phi $ is proper map, the convexity theorem tells us that

$$ \begin{align*}\Delta(\textbf{T}^{*}\tilde{K}\times \mathfrak{q}):=\mathrm{Image}(\Phi)\bigcap \, \tilde{\mathfrak{t}}^{*}_{\geq 0}\times \mathfrak{t}^{*}_{\geq 0} \end{align*} $$

is a closed convex locally polyhedral set.

We consider now the action of $\tilde {K}\times K$ on the affine variety $\tilde {K}_{\mathbb {C}}\times \mathfrak {q}$ . The set of highest weights of $\tilde {K}_{\mathbb {C}}\times \mathfrak {q}$ is the semigroup $\Pi ^{\mathbb {Z}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})\subset \tilde {\wedge }^{*}_{+}\times \wedge ^{*}_{+}$ consisting of all dominant weights $(\tilde {\lambda },\lambda )$ such that the irreducible $\tilde {K}\times K$ -representation $V_{\tilde {\lambda }}^{\tilde {K}}\otimes V_{\lambda }^{K}$ occurs in the coordinate ring $\mathbb {C}[\tilde {K}_{\mathbb {C}}\times \mathfrak {q}]$ . We denote by $\Pi ^{\mathbb {Q}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ the $\mathbb {Q}$ -convex cone generated by the semigroup $\Pi ^{\mathbb {Z}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ : $(\tilde {\xi },\xi )\in \Pi ^{\mathbb {Q}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ if and only if $\exists N\geq 1$ , $N(\tilde {\xi },\xi )\in \Pi ^{\mathbb {Z}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ .

The following important fact is classical (see Theorem 4.9 in [35]).

Proposition 4.4. The Kirwan polyhedron $\Delta (\textbf{T}^{*}\tilde {K}\times \mathfrak {q})$ is equal to the closure of the $\mathbb {Q}$ -convex cone $\Pi ^{\mathbb {Q}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ .

A direct application of the Peter–Weyl theorem gives the following characterization:

(16) $$ \begin{align} (\tilde{\lambda},\lambda)\in\Pi^{\mathbb{Z}}(\tilde{K}_{\mathbb{C}}\times \mathfrak{q})&\Longleftrightarrow \left[ V^{\tilde{K}}_{\tilde{\lambda}}\vert_{K}\otimes V^{K}_{\lambda}\otimes \mathrm{Sym}(\mathfrak{q}) \right]^{K}\neq 0\\ &\Longleftrightarrow \left[ V^{K}_{\lambda^{*}} : V^{\tilde{K}}_{\tilde{\lambda}}\vert_{K}\otimes \mathrm{Sym}(\mathfrak{q}) \right]\neq 0\nonumber\\ &\Longleftrightarrow (\tilde{\lambda},\lambda^{*})\in \Pi^{\mathbb{Z}}_{\mathfrak{q}}(\tilde{K},K).\nonumber \end{align} $$

4.3 Proof of Theorem B

Consider the semigroup $\Pi ^{\mathbb {Z}}_{\mathfrak {q}}(\tilde {K},K)$ of $\tilde {\wedge }^{*}_{+} \times \wedge ^{*}_{+}$ (see Definition 1.3) and the $\mathbb {Q}$ -convex cone $\Pi ^{\mathbb {Q}}_{\mathfrak {q}}(\tilde {K},K):=\{(\tilde {\xi },\xi )\in \tilde {\mathfrak {t}}^{*}_{\geq 0}\times \mathfrak {t}^{*}_{\geq 0} \; \ \exists N\geq 1, N(\tilde {\xi },\xi )\in \Pi ^{\mathbb {Z}}_{\mathfrak {q}}(\tilde {K},K)\}$ .

The Jakobsen–Vergne theorem says that $\Pi _{\mathrm {hol}}^{\mathbb {Z}}(\tilde {G},G)=\Pi ^{\mathbb {Z}}_{\mathfrak {q}}(\tilde {K},K)\,\bigcap \, \widehat {\tilde {G}}_{\mathrm {hol}}\times \widehat {G}_{\mathrm {hol}}$ . Hence, the convex cone $\Pi _{\mathrm {hol}}^{\mathbb {Q}}(\tilde {G},G)$ is equal to $\Pi ^{\mathbb {Q}}_{\mathfrak {q}}(\tilde {K},K)\cap \tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ . Thanks to equation (16), we know that $(\tilde {\xi },\xi )\in \Pi ^{\mathbb {Q}}_{\mathfrak {q}}(\tilde {K},K)$ if and only if $(\tilde {\xi },\xi ^{*})\in \Pi ^{\mathbb {Q}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ . The density results obtained in Proposition 4.4 and Corollary 4.3 gives finally Theorem B.

4.4 Proof of Theorem C

We denote by $\bar {\mathfrak {q}}$ the K-vector space $\mathfrak {q}$ equipped with the opposite symplectic form $-\Omega _{\mathfrak {q}}$ and opposite complex structure $-\mathrm {ad}(z)$ . The moment map relative to the K-action on $\bar {\mathfrak {q}}$ is denoted by $\Phi _{\bar {\mathfrak {q}}}=-\Phi _{\mathfrak {q}}$ .

Lemma 4.5. Any element $(\tilde {\xi },\xi )\in \tilde {\mathfrak {t}}^{*}_{\geq 0}\times \mathfrak {t}^{*}_{\geq 0}$ satisfies the equivalence

$$ \begin{align*}(\tilde{\xi},\xi^{*})\in \Delta(\textbf{T}^{*}\tilde{K}\times \mathfrak{q}) \Longleftrightarrow \xi\in \Delta_{K}(\tilde{K}\tilde{\xi}\times \overline{\mathfrak{q}}). \end{align*} $$

Proof. Thanks to equation (15), we see immediatly that $\exists (\tilde {a},\tilde {\eta },Y)\in \textbf { T}^{*}\tilde {K}\times \mathfrak {q}$ such that $(\tilde {\xi },\xi ^{*})=\Phi (\tilde {a},\tilde {\eta },Y)$ if and only if $\exists (\tilde {b},Z)\in \tilde {K}\times \mathfrak {q}$ such that $\xi =\pi _{\mathfrak {k},\tilde {\mathfrak {k}}}(\tilde {b}\tilde {\xi })+\Phi _{\bar {\mathfrak {q}}}(Z)$ .

At this stage, we know that $\Delta _{G}(\tilde {G}\tilde {\mu })=\Delta _{K}(\tilde {K}\tilde {\mu }\times \overline {\mathfrak {q}})\cap \mathcal {C}_{\mathrm {hol}}$ . Hence, Theorem C will follow from the next result.

Proposition 4.6. For any $\tilde {\mu }\in \tilde {\mathcal {C}}_{\mathrm {hol}}$ , the Kirwan polyhedron $\Delta _{K}(\tilde {K}\tilde {\mu }\times \overline {\mathfrak {q}})$ is contained in $\mathcal {C}_{\mathrm {hol}}$ .

Proof. By definition $\mathcal {C}_{\mathrm {hol}}=\mathcal {C}_{G/K}^{0}\cap \mathfrak {t}^{*}_{\geq 0}$ , so we have to prove that $\pi _{\mathfrak {k},\tilde {\mathfrak {k}}}(\tilde {K}\tilde {\mu })+ \mathrm {Image}( \Phi _{\bar {\mathfrak {q}}})$ is contained in $\mathcal {C}_{G/K}^{0}$ . By definition $\tilde {K}\tilde {\mu }\subset \mathcal {C}_{\tilde {G}/\tilde {K}}^{0}$ , and then $\pi _{\mathfrak {k},\tilde {\mathfrak {k}}}(\tilde {K}\tilde {\mu })\subset \mathcal {C}_{G/K}^{0}$ . Since $\mathcal {C}_{G/K}^{0} +\mathcal {C}_{G/K}\subset \mathcal {C}_{G/K}^{0}$ , it is sufficient to check that $\mathrm {Image}( \Phi _{\bar {\mathfrak {q}}})\subset \mathcal {C}_{G/K}$ . Let $\Phi _{\tilde {\mathfrak {p}}}$ be the moment map relative to the action of $\tilde {K}$ on $(\tilde {\mathfrak {p}},\Omega _{\tilde {\mathfrak {p}}})$ . As $\mathrm {Image}( \Phi _{\bar {\mathfrak {q}}})\subset \pi _{\mathfrak {k},\tilde {\mathfrak {k}}}\left (\mathrm {Image}( -\Phi _{\tilde {\mathfrak {p}}})\right )$ , the following lemma will terminate the proof of Proposition 4.6.

Lemma 4.7. The image of the moment map $-\Phi _{\tilde {\mathfrak {p}}}$ is contained in $\mathcal {C}_{\tilde {G}/\tilde {K}}$ .

Proof. Let $z^{*}\in \tilde {\mathfrak {t}}^{*}$ such that $\langle z^{*},\tilde {X}\rangle = -\tilde {\kappa }(z,\tilde {X})$ , $\forall \tilde {X}\in \tilde {\mathfrak {g}}$ . Consider the coadjoint orbit $\tilde {\mathcal {O}}=\tilde {G} z^{*}$ equipped with its canonical symplectic structure $\Omega _{\tilde {\mathcal {O}}}$ : The symplectic vector space $\textbf { T}_{z^{*}}\tilde {\mathcal {O}}$ is canonically isomorphic to $(\tilde {\mathfrak {p}},-\Omega _{\tilde {\mathfrak {p}}})$ . In [26], McDuff proved that $(\tilde {\mathcal {O}},\Omega _{\tilde {\mathcal {O}}})$ is diffeomorphic, as a $\tilde {K}$ -symplectic manifold, to the symplectic vector space $(\tilde {\mathfrak {p}},-\Omega _{\tilde {\mathfrak {p}}})$ (see [6, 8] for a generalization of this fact). McDuff’s results show in particular that $\mathrm {Image}(-\Phi _{\tilde {\mathfrak {p}}})=\pi _{\tilde {\mathfrak {g}},\tilde {\mathfrak {k}}}(\tilde {\mathcal {O}})$ . Our proof is completed if we check that $\pi _{\tilde {\mathfrak {g}},\tilde {\mathfrak {k}}}(\tilde {\mathcal {O}})\subset \mathcal {C}_{\tilde {G}/\tilde {K}}$ : In other words, if $\langle \pi _{\tilde {\mathfrak {g}},\tilde {\mathfrak {k}}}(\tilde {g}_{0}\, z^{*}),\tilde {g}_{1}z\rangle \geq 0$ , $\forall \tilde {g}_{0},\tilde {g}_{1}\in \tilde {G}$ . But

$$ \begin{align*} 2\langle\pi_{\tilde{\mathfrak{g}},\tilde{\mathfrak{k}}}(\tilde{g}_{0}\, z^{*}),\tilde{g}_{1}\,z\rangle &= \langle\tilde{g}_{0}\, z^{*},\tilde{g}_{1}z+\Theta(\tilde{g}_{1})z\rangle\\ &= -\tilde{\kappa}(z,\tilde{g}_{0}^{-1}\tilde{g}_{1}\, z)-\tilde{\kappa}(z,\tilde{g}_{0}^{-1}\Theta(\tilde{g}_{1})z). \end{align*} $$

With equation (7) in hand, it is not difficult to see that $-\tilde {\kappa }(z,\tilde {g}\, z)\geq 0$ for every $\tilde {g}\in \tilde {G}$ . We thus verified that $\pi _{\tilde {\mathfrak {g}},\tilde {\mathfrak {k}}}(\tilde {\mathcal {O}})\subset \mathcal {C}_{\tilde {G}/\tilde {K}}$ .

5 Inequalities characterizing the cones $\Delta _{\mathrm {hol}}(\tilde {G},G)$

We come back to the framework of §4.2. We consider the Kähler Hamiltonian $\tilde {K}\times K$ -manifold $\textbf { T}^{*}\tilde {K}\times \mathfrak {q}$ . The moment map, $\Phi : \textbf { T}^{*}\tilde {K}\times \mathfrak {q}\to \tilde {\mathfrak {k}}^{*}\oplus \mathfrak {k}^{*}$ , relative to the $\tilde {K}\times K$ -action, is defined by equation (15).

In this section, we adapt to our case the result of §6 of [32] concerning the parametrization of the facets of Kirwan polyhedrons in terms of Ressayre’s data.

5.1 Admissible elements

We choose maximal tori $\tilde {T}\subset \tilde {K}$ and $T\subset K$ such that $T\subset \tilde {T}$ . Let $\mathfrak {R}_{o}$ and $\mathfrak {R}$ be, respectively, the set of roots for the action of T on $(\tilde {\mathfrak {g}}/\mathfrak {g})\otimes \mathbb {C}$ and $\mathfrak {g}\otimes \mathbb {C}$ . Let $\tilde {\mathfrak {R}}$ be the set of roots for the action of $\tilde {T}$ on $\tilde {\mathfrak {g}}\otimes \mathbb {C}$ . Let $\mathfrak {R}^{+}\subset \mathfrak {R}$ and $\tilde {\mathfrak {R}}^{+}\subset \tilde {\mathfrak {R}}$ be the systems of positive roots defined in equation (6). Let $W,\tilde {W}$ be the Weyl groups of $(T,K)$ and $(\tilde {T},\tilde {K})$ . Let $w_{o}\in W$ be the longest element.

We start by introducing the notion of admissible elements. The group $\hom (U(1),T)$ admits a natural identification with the lattice $\wedge :=\frac {1}{2\pi }\ker (\exp :\mathfrak {t}\to T)$ . A vector $\gamma \in \mathfrak {t}$ is called rational if it belongs to the $\mathbb {Q}$ -vector space $\mathfrak {t}_{\mathbb {Q}}$ generated by $\wedge $ .

We consider the $\tilde {K}\times K$ -action on $N:=\textbf { T}^{*} \tilde {K} \times \mathfrak {q}$ . We associate to any subset $\mathcal {X}\subset N$ , the integer $\dim _{\tilde {K}\times K}(\mathcal {X})$ (see equation (5)).

Definition 5.1. A nonzero element $(\tilde {\gamma },\gamma )\in \tilde {\mathfrak {t}}\times \mathfrak {t}$ is called admissible if the elements $\tilde {\gamma }$ and $\gamma $ are rational and if $\dim _{\tilde {K}\times K}(N^{(\tilde {\gamma },\gamma )})-\dim _{\tilde {K}\times K}(N)\in \{0,1\}$ .

If $\gamma \in \mathfrak {t}$ , we denote by $\mathfrak {R}_{o}\cap \gamma ^{\perp }$ the subsets of weight vanishing against $\gamma $ . We start with the following lemma whose proof is left to the reader (see §6.1.1 of [32]).

Lemma 5.2.

1. 1. $N^{(\tilde {\gamma },\gamma )}\neq \emptyset $ if and only if $\tilde {\gamma }\in \tilde {W}\gamma $ .

2. 2. $\dim _{\tilde {K}\times K}(N)=\dim _{T}(\tilde {\mathfrak {g}}/\mathfrak {g})=\dim (\mathfrak {t})-\dim (\mathrm {Vect}(\mathfrak {R}_{o}))$ .

3. 3. For any $\tilde {w}\in \tilde {W}$ , $\dim _{\tilde {K}\times K}(N^{(\tilde {w}\gamma ,\gamma )})= \dim _{T}(\tilde {\mathfrak {g}}^{\gamma }/\mathfrak {g}^{\gamma })=\dim (\mathfrak {t})-\dim (\mathrm {Vect}(\mathfrak {R}_{o}\cap \gamma ^{\perp }))$ .

The next result is a direct consequence of the previous lemma.

Lemma 5.3. The admissible elements relative to the $\tilde {K}\times K$ -action on $\textbf { T}^{*} \tilde {K} \times \mathfrak {q}$ are of the form $(\tilde {w}\gamma ,\gamma )$ , where $\tilde {w}\in \tilde {W}$ and $\gamma $ is a nonzero rational element satisfying $\mathrm {Vect}(\mathfrak {R}_{o})\cap \gamma ^{\perp }=\mathrm {Vect}(\mathfrak {R}_{o}\cap \gamma ^{\perp })$ .

5.2 Ressayre’s data

Definition 5.4.

1. 1. Consider the linear action $\rho : G\to \mathrm {GL}_{\mathbb {C}}(V)$ of a compact Lie group on a complex vector space V. For any $(\eta ,a)\in \mathfrak {g}\times \mathbb {R}$ , we define the vector subspace $V^{\eta =a}=\{v\in V, d\rho (\eta )v=i av\}$ . Thus, for any $\eta \in \mathfrak {g}$ , we have the decomposition $V=V^{\eta>0}\oplus V^{\eta =0}\oplus V^{\eta <0}$ , where $V^{\eta>0}=\sum _{a>0}V^{\eta =a}$ , and $V^{\eta <0}=\sum _{a<0}V^{\eta =a}$ .

2. 2. The real number $\operatorname {Tr}_{\eta }(V^{\eta>0})$ is defined as the sum $\sum _{a>0}a\,\dim (V^{\eta =a})$ .

We consider an admissible element $(\tilde {w}\gamma ,\gamma )$ . The submanifold of $N\simeq \tilde {K}_{\mathbb {C}} \times \mathfrak {q}$ fixed by $(\tilde {w}\gamma ,\gamma )$ is $N^{(\tilde {w}\gamma ,\gamma )}= \tilde {w} \tilde {K}_{\mathbb {C}}^{\gamma } \times \mathfrak {q}^{\gamma }$ . There is a canonical isomorphism between the manifold $N^{(\tilde {w}\gamma ,\gamma )}$ equipped with the action of $\tilde {w}\tilde {K}^{\gamma } \tilde {w}^{-1}\times K^{\gamma }$ with the manifold $\tilde {K}_{\mathbb {C}}^{\gamma } \times \mathfrak {q}^{\gamma }$ equipped with the action of $\tilde {K}^{\gamma } \times K^{\gamma }$ . The tangent bundle $(\textbf { T} N\vert _{N^{(\tilde {w}\gamma ,\gamma )}})^{(\tilde {w}\gamma ,\gamma )>0}$ is isomorphic to $N^{\gamma _{w}}\times \tilde {\mathfrak {k}}_{\mathbb {C}}^{\gamma>0}\times \mathfrak {q}^{\gamma >0}$ .

The choice of positive roots $\mathfrak {R}^{+}$ (resp. $\tilde {\mathfrak {R}}^{+}$ ) induces a decomposition $\mathfrak {k}_{\mathbb {C}}=\mathfrak {n}\oplus \mathfrak {t}_{\mathbb {C}}\oplus \overline {\mathfrak {n}}$ (resp. $\tilde {\mathfrak {k}}_{\mathbb {C}}=\tilde {\mathfrak {n}}\oplus \tilde {\mathfrak {t}}_{\mathbb {C}}\oplus \overline {\tilde {\mathfrak {n}}}$ ), where $\mathfrak {n}=\sum _{\alpha \in \mathfrak {R}^{+}}(\mathfrak {k}\otimes \mathbb {C})_{\alpha }$ (resp. $\tilde {\mathfrak {n}}=\sum _{\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}}(\tilde {\mathfrak {k}}\otimes \mathbb {C})_{\tilde {\alpha }}$ ). We consider the map

$$ \begin{align*}\rho^{\tilde{w},\gamma}: \tilde{K}_{\mathbb{C}}^{\gamma} \times \mathfrak{q}^{\gamma} \longrightarrow \hom \left(\tilde{\mathfrak{n}}^{\tilde{w}\gamma>0}\times \mathfrak{n}^{\gamma>0}\,,\, \tilde{\mathfrak{k}}_{\mathbb{C}}^{\gamma>0}\times\mathfrak{q}^{\gamma>0}\right) \end{align*} $$

defined by the relation

$$ \begin{align*}\rho^{\tilde{w},\gamma}(\tilde{x},v): (\tilde{X},X)\longmapsto ((\tilde{w} \tilde{x})^{-1}\tilde{X}-X\, ;\, X\cdot v) \end{align*} $$

for any $(\tilde {x},v)\in \tilde {K}_{\mathbb {C}}^{\gamma } \times \mathfrak {q}^{\gamma }$ .

Definition 5.5. $(\gamma ,\tilde {w})\in \mathfrak {t}\times \tilde {W}$ is a Ressayre’s datum if

1. 1. $(\tilde {w}\gamma ,\gamma )$ is admissible,

2. 2. $\exists (\tilde {x},v)$ such that $\rho ^{\tilde {w},\gamma }(\tilde {x},v)$ is bijective.

Remark 5.6. In [32], the Ressayre’s data were called regular infinitesimal B-Ressayre’s pairs.

Since the linear map $\rho ^{\tilde {w},\gamma }(\tilde {x},v)$ commutes with the $\gamma $ -actions, we obtain the following necessary conditions.

Lemma 5.7. If $(\gamma ,\tilde {w})\in \mathfrak {t}\times \tilde {W}$ is a Ressayre’s datum, then

• • Relation (A): $\dim (\tilde {\mathfrak {n}}^{\tilde {w}\gamma>0})+\dim (\mathfrak {n}^{\gamma >0})=\dim (\tilde {\mathfrak {k}}_{\mathbb {C}}^{\gamma >0})+\dim (\mathfrak {q}^{\gamma >0})$ .

• • Relation (B): $\operatorname {Tr}_{\tilde {w}\gamma }(\tilde {\mathfrak {n}}^{\tilde {w}\gamma>0})+\operatorname {Tr}_{\gamma }(\mathfrak {n}^{\gamma >0})= \operatorname {Tr}_{\gamma }(\tilde {\mathfrak {k}}_{\mathbb {C}}^{\gamma >0})+\operatorname {Tr}_{\gamma }(\mathfrak {q}^{\gamma >0})$ .

Lemma 5.8. Relation (B) is equivalent to

(17) $$ \begin{align} \sum_{\stackrel{\alpha\in\mathfrak{R}^{+}}{\langle\alpha,\gamma\rangle> 0}}\langle\alpha,\gamma\rangle= \sum_{\stackrel{\tilde{\alpha}\in\tilde{\mathfrak{R}}^{+}}{\langle\tilde{\alpha},\tilde{w}_{0}\tilde{w}\gamma\rangle> 0}}\langle\tilde{\alpha},\tilde{w}_{0}\tilde{w}\gamma\rangle. \end{align} $$

Proof. First, one sees that $\operatorname {Tr}_{\gamma }(\mathfrak {q}^{\gamma>0})=\operatorname {Tr}_{\gamma }(\tilde {\mathfrak {p}}^{\gamma >0})-\operatorname {Tr}_{\gamma }(\mathfrak {p}^{\gamma >0})= \sum _{\stackrel {\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}_{n}}{\langle \tilde {\alpha },\gamma \rangle >0}}\langle \tilde {\alpha },\gamma \rangle -\sum _{\stackrel {\alpha \in \mathfrak {R}^{+}_{n}}{\langle \alpha ,\gamma \rangle >0}}\langle \alpha ,\gamma \rangle $ , and $\operatorname {Tr}_{\gamma }(\tilde {\mathfrak {k}}_{\mathbb {C}}^{\gamma>0})=\operatorname {Tr}_{\tilde {w}\gamma }(\tilde {\mathfrak {k}}_{\mathbb {C}}^{\tilde {w}\gamma >0})=\operatorname {Tr}_{\tilde {w}\gamma }(\tilde {\mathfrak {n}}^{\tilde {w}\gamma >0})+ \sum _{\stackrel {\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}_{c}}{\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle >0}}\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle $ . Relation $(B)$ is equivalent to

(18) $$ \begin{align} \operatorname{Tr}_{\gamma}(\mathfrak{n}^{\gamma>0})+\sum_{\stackrel{\alpha\in \mathfrak{R}^{+}_{n}}{\langle\alpha,\gamma\rangle>0}}\langle\alpha,\gamma\rangle= \sum_{\stackrel{\tilde{\alpha}\in \tilde{\mathfrak{R}}^{+}_{n}}{\langle\tilde{\alpha},\gamma\rangle>0}}\langle\tilde{\alpha},\gamma\rangle +\sum_{\stackrel{\tilde{\alpha}\in \tilde{\mathfrak{R}}^{+}_{c}}{\langle\tilde{\alpha},\tilde{w}_{0}\tilde{w}\gamma\rangle>0}}\langle\tilde{\alpha},\tilde{w}_{0}\tilde{w}\gamma\rangle. \end{align} $$

Since $\tilde {\mathfrak {R}}^{+}_{n}$ is invariant under the action of the Weyl group $\tilde {W}$ , the right-hand side of equation (18) is equal to $\sum _{\stackrel {\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}}{\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle>0}}\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle $ . Since the left-hand side of equation (18) is equal to $\sum _{\stackrel {\alpha \in \mathfrak {R}^{+}}{\langle \alpha ,\gamma \rangle>0}}\langle \alpha ,\gamma \rangle $ , the proof of the lemma is complete.

5.3 Cohomological characterization of Ressayre’s data

Let $\gamma \in \mathfrak {t}$ be a nonzero rational element. We denote by $B\subset K_{\mathbb {C}}$ and by $\tilde {B}\subset \tilde {K}_{\mathbb {C}}$ the Borel subgroups with Lie algebra $\mathfrak {b}=\mathfrak {t}_{\mathbb {C}}\oplus \mathfrak {n}$ and $\tilde {\mathfrak {b}}=\tilde {\mathfrak {t}}_{\mathbb {C}}\oplus \tilde {\mathfrak {n}}$ . Consider the parabolic subgroup $P_{\gamma }\subset K_{\mathbb {C}}$ defined by

(19) $$ \begin{align} P_{\gamma}=\{g\in K_{\mathbb{C}}, \lim_{t\to\infty}\exp(-it\gamma)g\exp(it\gamma)\ \mathrm{exists}\}. \end{align} $$

Similarly, one defines a parabolic subgroup $\tilde {P}_{\gamma }\subset \tilde {K}_{\mathbb {C}}$ .

We work with the projective varieties $\mathcal {F}_{\gamma }:=K_{\mathbb {C}}/P_{\gamma }$ , $\tilde {\mathcal {F}}_{\gamma }:=\tilde {K}_{\mathbb {C}}/\tilde {P}_{\gamma }$ and the canonical embedding $\iota : \mathcal {F}_{\gamma }\to \tilde {\mathcal {F}}_{\gamma }$ . We associate to any $\tilde {w}\in \tilde {W}$ , the Schubert cell

$$ \begin{align*}\tilde{\mathfrak{X}}^{o}_{\tilde{w},\gamma}:= \tilde{B}[\tilde{w}]\subset \tilde{\mathcal{F}}_{\gamma} \end{align*} $$

and the Schubert variety $\tilde {\mathfrak {X}}_{\tilde {w},\gamma }:=\overline {\tilde {\mathfrak {X}}^{o}_{\tilde {w},\gamma }}$ . If $\tilde {W}^{\gamma }$ denotes the subgroup of $\tilde {W}$ that fixes $\gamma $ , we see that the Schubert cell $\tilde {\mathfrak {X}}^{o}_{\tilde {w},\gamma }$ and the Schubert variety $\tilde {\mathfrak {X}}_{\tilde {w},\gamma }$ depend only of the class of $\tilde {w}$ in $\tilde {W}/\tilde {W}^{\gamma }$ .

On the variety $\mathcal {F}_{\gamma }$ , we consider the Schubert cell $\mathfrak {X}^{o}_{\gamma }:= B[e]$ and the Schubert variety $\mathfrak {X}_{\gamma }:=\overline {\mathfrak {X}^{o}_{\gamma }}$ .

We consider the cohomology ¹ ring $H^{*}(\tilde {\mathcal {F}}_{\gamma },\mathbb {Z})$ of $\tilde {\mathcal {F}}_{\gamma }$ . If Y is an irreducible closed subvariety of $\tilde {\mathcal {F}}_{\gamma }$ , we denote by $[Y]\in H^{2n_{Y}}(\tilde {\mathcal {F}}_{\gamma },\mathbb {Z})$ its cycle class in cohomology: Here $n_{Y}=\mathrm {codim}_{\mathbb {C}}(Y)$ . Let $\iota ^{*}:H^{*}(\tilde {\mathcal {F}}_{\gamma },\mathbb {Z})\to H^{*}(\mathcal {F}_{\gamma },\mathbb {Z})$ be the pull-back map in cohomology. Recall that the cohomology class $[pt]$ associated to a singleton $Y=\{pt\}\subset \mathcal {F}_{\gamma }$ is a basis of $H^{\mathrm {max}}(\mathcal {F}_{\gamma },\mathbb {Z})$ .

To an oriented real vector bundle $\mathcal {E}\to N$ of rank r, we can associate its Euler class $\mathrm {Eul}(\mathcal {E})\in H^{2r}(N,\mathbb {Z})$ . When $\mathcal {V}\to N$ is a complex vector bundle, then $\mathrm {Eul}(\mathcal {V}_{\mathbb {R}})$ corresponds to the top Chern class $c_{p}(\mathcal {V})$ , where p is the complex rank of $\mathcal {V}$ , and $\mathcal {V}_{\mathbb {R}}$ means $\mathcal {V}$ viewed as a real vector bundle oriented by its complex structure (see [5], §21).

The isomorphism $\mathfrak {q}^{\gamma>0}\simeq \mathfrak {q}/ \mathfrak {q}^{\gamma \leq 0}$ shows that $\mathfrak {q}^{\gamma>0}$ can be viewed as a $P_{\gamma }$ -module. Let $[\mathfrak {q}^{\gamma>0}]= K_{\mathbb {C}}\times _{P_{\gamma }}\mathfrak {q}^{\gamma >0}$ be the corresponding complex vector bundle on $\mathcal {F}_{\gamma }$ . We denote simply by $\mathrm {Eul}(\mathfrak {q}^{\gamma>0})$ the Euler class $\mathrm {Eul}([\mathfrak {q}^{\gamma>0}]_{\mathbb {R}})\in H^{*}(\mathcal {F}_{\gamma },\mathbb {Z})$ .

The following characterization of Ressayre’s data was obtained in [32], §6. Recall that $\mathfrak {R}_{o}$ denotes the set of weights relative to the T-action on $(\tilde {\mathfrak {g}}/\mathfrak {g})\otimes \mathbb {C}$ .

Proposition 5.9. An element $(\gamma ,\tilde {w})\in \mathfrak {t}\times \tilde {W}$ is a Ressayre’s datum if and only if the following conditions hold:

• • $\gamma $ is nonzero and rational.

• • $\mathrm {Vect}(\mathfrak {R}_{o}\cap \gamma ^{\perp })=\mathrm {Vect}(\mathfrak {R}_{o})\cap \gamma ^{\perp }$ .

• • $[\mathfrak {X}_{\gamma }]\cdot \iota ^{*}([\tilde {\mathfrak {X}}_{\tilde {w},\gamma }])\cdot \mathrm {Eul}(\mathfrak {q}^{\gamma>0})= k[pt],\ k\geq 1$ in $H^{*}(\mathcal {F}_{\gamma },\mathbb {Z})$ .

• • $\sum _{\stackrel {\alpha \in \mathfrak {R}^{+}}{\langle \alpha ,\gamma \rangle> 0}}\langle \alpha ,\gamma \rangle = \sum _{\stackrel {\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}}{\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle > 0}}\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle $ .

5.4 Parametrization of the facets

We can finally describe the Kirwan polyhedron $\Delta (\textbf{T}^{*}\tilde {K}\times \mathfrak {q})$ (see [32], § 6).

Theorem 5.10. An element $(\tilde {\xi },\xi )\in \tilde {\mathfrak {t}}_{\geq 0}^{*}\times \mathfrak {t}_{\geq 0}^{*}$ belongs to $\Delta (\textbf{T}^{*}\tilde {K}\times \mathfrak {q})$ if and only if

$$ \begin{align*}\langle \tilde{\xi},\tilde{w}\gamma\rangle+\langle \xi,\gamma\rangle\geq 0 \end{align*} $$

for any Ressayre’s datum $(\gamma ,\tilde {w})\in \mathfrak {t}\times \tilde {W}$ .

Theorem 5.10 and Theorem B permit us to give the following description of the convex cone $\Delta _{\mathrm {hol}}(\tilde {G},G)$ .

Theorem 5.11. An element $(\tilde {\xi },\xi )$ belongs to $\Delta _{\mathrm {hol}}(\tilde {G},G)$ if and only if $(\tilde {\xi },\xi )\in \tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ and

$$ \begin{align*}\langle \tilde{\xi},\tilde{w}\gamma\rangle\geq \langle \xi,w_{0}\gamma\rangle \end{align*} $$

for any $(\gamma ,\tilde {w})\in \mathfrak {t}\times \tilde {W}$ satisfying the following conditions:

• • $\gamma $ is nonzero and rational.

• • $\mathrm {Vect}(\mathfrak {R}_{o}\cap \gamma ^{\perp })=\mathrm {Vect}(\mathfrak {R}_{o})\cap \gamma ^{\perp }$ .

• • $[\mathfrak {X}_{\gamma }]\cdot \iota ^{*}([\tilde {\mathfrak {X}}_{\tilde {w},\gamma }])\cdot \mathrm {Eul}(\mathfrak {q}^{\gamma>0})= k[pt],\ k\geq 1$ in $H^{*}(\mathcal {F}_{\gamma },\mathbb {Z})$ .

• • $\sum _{\stackrel {\alpha \in \mathfrak {R}^{+}}{\langle \alpha ,\gamma \rangle> 0}}\langle \alpha ,\gamma \rangle = \sum _{\stackrel {\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}}{\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle > 0}}\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle $ .

6 Example: the holomorphic Horn cone $\mathrm {Horn}_{\mathrm {hol}}(p,q)$

Let $p\geq q\geq 1$ . We consider the pseudo-unitary group $G=U(p,q)\subset GL_{p+q}(\mathbb {C})$ defined by the relation: $g\in U(p,q)$ if and only if $g\mathrm {Id}_{p,q}g^{*}=\mathrm {Id}_{p,q}$ , where $\mathrm {Id}_{p,q}$ is the diagonal matrix $\mathrm {Diag}(\mathrm {Id}_{p},-\mathrm {Id}_{q})$ .

We work with the maximal compact subgroup $K=U(p)\times U(q)\subset G$ . We have the Cartan decomposition $\mathfrak {g}=\mathfrak {k}\oplus \mathfrak {p}$ , where $\mathfrak {p}$ is identified with the vector space $M_{p,q}$ of $p\times q$ matrices through the map

$$ \begin{align*}X\in M_{p,q}\longmapsto\left(\begin{array}{@{}cc@{}} 0 & X \\ X^{*} & 0\\\end{array}\right). \end{align*} $$

We work with the element $z_{p,q}=\frac {i}{2}\mathrm {Id}_{p,q}$ which belongs to the center of $\mathfrak {k}$ . The adjoint action of $z_{p,q}$ on $\mathfrak {p}$ corresponds to the standard complex structure on $M_{p,q}$ .

The trace on $\mathfrak {gl}_{p+q}(\mathbb {C})$ defines an identification $\mathfrak {g}\simeq \mathfrak {g}^{*}=\hom (\mathfrak {g},\mathbb {R})$ : To $X\in \mathfrak {g}$ we associate $\xi _{X}\in \mathfrak {g}^{*}$ defined by $\langle \xi _{X},Y\rangle =-\mathrm {Tr}(XY)$ . Thus, the G-invariant cone $\mathcal {C}_{G/K}$ defined by $z_{p,q}$ can be viewed as the following cone of $\mathfrak {g}$ :

$$ \begin{align*}\mathcal{C}(p,q)=\left\{X\in\mathfrak{g},\ \mathrm{Im}\left(\operatorname{Tr}( gXg^{-1} \mathrm{Id}_{p,q})\right) \geq 0,\ \forall g\in U(p,q)\right\}. \end{align*} $$

Let $T\subset U(p)\times U(q)$ be the maximal torus formed by the diagonal matrices. The Lie algebra $\mathfrak {t}$ is identified with $\mathbb {R}^{p}\times \mathbb {R}^{q}$ through the map $\mathbf {d}:\mathbb {R}^{p}\times \mathbb {R}^{q}\to \mathfrak {u}(p)\times \mathfrak {u}(q)$ : $\mathbf {d}_{x}=\mathrm {Diag}(ix_{1},\cdots ,ix_{p},i x_{p+1},\cdots , i x_{p+q})$ . The Weyl chamber is

$$ \begin{align*}\mathfrak{t}_{\geq 0}=\left\{x\in \mathbb{R}^{p}\times\mathbb{R}^{q},\ x_{1}\geq \cdots \geq x_{p} \ \mathrm{and}\ x_{p+1}\geq \cdots \geq x_{p+q}\right\}. \end{align*} $$

Proposition 2.2 tells us that the $U(p,q)$ adjoint orbits in the interior of $\mathcal {C}(p,q)$ are parametrized by the holomorphic chamber

$$ \begin{align*}\mathcal{C}_{p,q}=\left\{x\in \mathbb{R}^{p}\times\mathbb{R}^{q}, x_{1}\geq \cdots \geq x_{p}> x_{p+1}\geq \cdots \geq x_{p+q}\right\} \,\subset\, \mathfrak{t}_{\geq 0}. \end{align*} $$

Definition 6.1. The holomorphic Horn cone $\mathrm {Horn}_{\mathrm {hol}}(p,q):=\mathrm {Horn}_{\mathrm {hol}}^{2}(U(p,q))$ is defined by the relations

$$ \begin{align*}\mathrm{Horn}_{\mathrm{hol}}(p,q)=\left\{(A,B,C)\in(\mathcal{C}_{p,q})^{3},\ U(p,q)\textbf{d}_{C}\subset U(p,q)\textbf{d}_{A}+U(p,q)\textbf{d}_{B}\right\}. \end{align*} $$

Let us detail the description given of $\mathrm {Horn}_{\mathrm {hol}}(p,q)$ by Theorem B. For any $n\geq 1$ , we consider the semigroup $\wedge _{n}^{+}=\{(\lambda _{1}\geq \cdots \geq \lambda _{n})\}\subset \mathbb {Z}^{n}$ . If $\lambda =(\lambda ^{\prime },\lambda ^{\prime \prime })\in \wedge _{p}^{+}\times \wedge _{q}^{+}$ , then $V_{\lambda }:=V^{U(p)}_{\lambda ^{\prime }}\otimes V^{U(q)}_{\lambda ^{\prime \prime }}$ denotes the irreducible representation of $U(p)\times U(q)$ with highest weight $\lambda $ . We denote by $\mathrm {Sym}(M_{p,q})$ the symmetric algebra of $M_{p,q}$ .

Definition 6.2.

1. 1. $\mathrm { Horn}^{\mathbb {Z}}(p,q)$ is the semigroup of $(\wedge _{p}^{+}\times \wedge _{q}^{+})^{3}$ defined by the conditions:

   $$ \begin{align*}(\lambda,\mu,\nu)\in \mathrm{ Horn}^{\mathbb{Z}}(p,q)\Longleftrightarrow \left[V_{\nu}\,:\,V_{\lambda}\otimes V_{\mu}\otimes \mathrm{Sym}(M_{p,q})\right]\neq 0. \end{align*} $$

2. 2. $\mathrm { Horn}(p,q)$ is the convex cone of $(\mathfrak {t}_{\geq 0})^{3}$ defined as the closure of $\mathbb {Q}^{>0}\cdot \mathrm { Horn}^{\mathbb {Z}}(p,q)$ .

Theorem B asserts that

(20) $$ \begin{align} \mathrm{Horn}_{\mathrm{hol}}(p,q)=\mathrm{ Horn}(p,q)\,\bigcap \,(\mathcal{C}_{p,q})^{3}. \end{align} $$

In another article [33], we obtained a recursive description of the cones $\mathrm { Horn}(p,q)$ . This allows us to give the following description of the holomorphic Horn cone $\mathrm {Horn}_{\mathrm {hol}}(2,2)$ .

Example 6.3. An element $(A,B,C)\in (\mathbb {R}^{4})^{3}$ belongs to $\mathrm {Horn}_{\mathrm {hol}}(2,2)$ if and only if the following conditions hold:

$$ \begin{align*} \boxed{ \begin{array}{rcl} a_{1} \geq a_{2} &> & a_{3} \geq a_{4}\\ b_{1} \geq b_{2} & > & b_{3} \geq b_{4}\\ c_{1} \geq c_{2} & > & c_{3} \geq c_{4} \end{array} } \end{align*} $$

$$ \begin{align*}\boxed{a_{1}+a_{2}+a_{3}+a_{4}+b_{1}+b_{2}+b_{3}+b_{4}=c_{1}+c_{2}+c_{3}+c_{4}} \end{align*} $$

$$ \begin{align*}\boxed{a_{1}+a_{2}+b_{1}+b_{2}\leq c_{1}+c_{2}} \end{align*} $$

$$ \begin{align*} \boxed{ \begin{array}{rcl} a_{2}+b_{2}&\leq & c_{2}\\ a_{2}+b_{1}&\leq & c_{1}\\ a_{1}+b_{2}&\leq & c_{1} \end{array} } \end{align*} $$

$$ \begin{align*} \boxed{ \begin{array}{rcl} a_{3}+b_{3} &\geq & c_{3}\\ a_{3}+b_{4} &\geq & c_{4}\\ a_{4}+b_{3} &\geq & c_{4} \end{array} } \end{align*} $$

$$ \begin{align*} \boxed{ \begin{array}{rcl} a_{2}+a_{4}+b_{2}+b_{4} &\leq & c_{1}+c_{4} \\ a_{2}+a_{4}+b_{2}+b_{4} &\leq & c_{2}+c_{3}\\ a_{2}+a_{4}+b_{1}+b_{4} &\leq & c_{1}+c_{3}\\ a_{1}+a_{4}+b_{2}+b_{4} &\leq & c_{1}+c_{3}\\ a_{2}+a_{4}+b_{2}+b_{3} &\leq & c_{1}+c_{3}\\ a_{2}+a_{3}+b_{2}+b_{4} &\leq & c_{1}+c_{3} \end{array} } \end{align*} $$

7 A conjectural symplectomorphism

Let $\tilde {\mu }\in \tilde {\mathcal {C}}_{\mathrm {hol}}$ . In this section, we are interested in the geometry of the coadjoint orbit $\tilde {G}\tilde {\mu }$ viewed as a Hamiltonian G-manifold with proper moment map $\Phi ^{\tilde {\mu }}_{G}: \tilde {G}\tilde {\mu }\to \mathfrak {g}^{*}$ .

We start with a decomposition that we have already used. The pullback $Y_{\tilde {\mu }}=(\Phi ^{\tilde {\mu }}_{G})^{-1}(\mathfrak {k}^{*})$ is a symplectic submanifold of $\tilde {G}\tilde {\mu }$ which is stable under the K-action: Let $\Omega _{\tilde {\mu }}$ be the corresponding two form on $Y_{\tilde {\mu }}$ . The action of K on $(Y_{\tilde {\mu }},\Omega _{\tilde {\mu }})$ is Hamiltonian, with a proper moment map $\Phi ^{\tilde {\mu }}_{K}: Y_{\tilde {\mu }}\to \mathfrak {k}^{*}$ equal to the restriction of $\Phi ^{\tilde {\mu }}_{G}$ to $Y_{\tilde {\mu }}$ .

The map $[g,x]\mapsto gx$ defines a symplectomorphism

(21) $$ \begin{align} G\times_{K} Y_{\tilde{\mu}}\simeq \tilde{G}\tilde{\mu} \end{align} $$

so that $\Phi ^{\tilde {\mu }}_{G}([g,x])=g\cdot \Phi ^{\tilde {\mu }}_{K}(x)$ [31]. This allows us to see that the Kirwan polytope $\Delta _{G}(\tilde {G}\tilde {\mu })$ relative to the G-action on $\tilde {G}\tilde {\mu }$ is equal to the Kirwan polytope $\Delta _{K}(Y_{\tilde {\mu }})$ relative to the K-action on $Y_{\tilde {\mu }}$ .

We consider the orthogonal decomposition $\tilde {\mathfrak {p}}=\mathfrak {p}\oplus \mathfrak {q}$ . Mostow’s decomposition theorem [27] says that the map $\psi :\mathfrak {p}\times \mathfrak {q}\times \tilde {K}\to \tilde {G}$ , $(X,Y,\tilde {k})\mapsto e^{X} e^{Y}\tilde {k}$ is a diffeomorphism. This leads to the following result.

Lemma 7.1. We have the following G-equivariant diffeomorphisms:

$$ \begin{align*} \psi_{o}: G\times_{K}\left( \mathfrak{q}\times \tilde{K}\right)&\longrightarrow \tilde{G}\\ \left[ g; Y,\tilde{k} \right] &\longmapsto g e^{Y} \tilde{k}, \end{align*} $$

$$ \begin{align*} \psi_{\tilde{\mu}}: G\times_{K}\left(\mathfrak{q}\times \tilde{K}\tilde{\mu}\right)&\longrightarrow \tilde{G}\tilde{\mu}\\ \left[ g; Y,\xi \right] &\longmapsto g e^{Y} \xi. \end{align*} $$

We obtain the following geometric information on the K-manifold $Y_{\tilde {\mu }}$ .

Corollary 7.2. There exists a K-equivariant diffeomorphism $\mathfrak {q} \times \tilde {K}\tilde {\mu }\simeq Y_{\tilde {\mu }}$ .

Proof. Thanks to the diffeomorphisms (21) and $\psi _{\tilde {\mu }}$ , we know that the manifolds $G\times _{K} Y_{\tilde {\mu }}$ and $G\times _{K}(\mathfrak {q}\times \tilde {K}\tilde {\mu })$ admit a G-equivariant diffeomorphism. Our result follows from this.

Let $\tilde {\kappa }$ be the Killing form on the Lie algebra $\tilde {\mathfrak {g}}$ . We consider the $\tilde {K}$ -invariant symplectic structures $\Omega _{\tilde {\mathfrak {p}}}$ on $\tilde {\mathfrak {p}}$ , defined by the relation $\Omega _{\tilde {\mathfrak {p}}}(\tilde {Y},\tilde {Y}^{\prime })=\tilde {\kappa }(z,[\tilde {Y},\tilde {Y}^{\prime }]),\ \forall \tilde {Y},\tilde {Y}^{\prime }\in \tilde {\mathfrak {p}}$ . We denote by $\Omega _{\mathfrak {q}}$ the restriction of $\Omega _{\tilde {\mathfrak {p}}}$ on the symplectic subspace $\mathfrak {q}$ .

We consider the following symplectic structure $-\Omega _{\mathfrak {q}}\times \Omega _{\tilde {K}\tilde {\mu }}$ on $\mathfrak {q} \times \tilde {K}\tilde {\mu }$ . Knowing that $\Delta _{G}(\tilde {G}\tilde {\mu })=\Delta _{K}(Y_{\tilde {\mu }})$ , the following conjectural result would give another proof of Theorem C.

Conjecture 7.3. There exists a K-equivariant symplectomorphism between $(Y_{\tilde {\mu }},\Omega _{\tilde {\mu }})$ and $(\mathfrak {q} \times \tilde {K}\tilde {\mu },-\Omega _{\mathfrak {q}}\times \Omega _{\tilde {K}\tilde {\mu }})$ .

This conjecture generalizes some results obtained when $G=\tilde {K}$ :

1. 1. In [26], McDuff showed that $\tilde {G}\tilde {\mu }\simeq \tilde {G}/\tilde {K}$ admit a $\tilde {K}$ -equivariant symplectomorphism with $(\tilde {\mathfrak {p}},-\Omega _{\tilde {\mathfrak {p}}})$ when $\tilde {\mu }$ is a central element of $\tilde {\mathfrak {k}}^{*}$ .

2. 2. In [8], Deltour extended the result of McDuff by showing that $\tilde {G}\tilde {\mu }$ admits a $\tilde {K}$ -equivariant symplectomorphism with $(\tilde {\mathfrak {p}}\times \tilde {K}\tilde {\mu },-\Omega _{\tilde {\mathfrak {p}}}\times \Omega _{\tilde {K}\tilde {\mu }})$ for any $\tilde {\mu }\in \tilde {\mathcal {C}}_{\mathrm {hol}}$ .