CHARACTER OF THE OSCILLATOR REPRESENTATION

Jeffrey Adams

§1. Introduction.

The oscillator representation ω of the metaplectic group S̃p(2n, ℝ) is the smallest representation of S̃p(2n, ℝ) which is genuine, i.e. does not factor to Sp(2n, ℝ). The main result of this paper is a short computation of the global character of ω. Formally the argument is very simple and we sketch it in the introduction. We will be more precise starting in §2.

Let π_{DS} be the holomorphic discrete series representation of S̃p(2n, ℝ) with the same infinitesimal character as that of the oscillator representation. Let ω = ω_{even} ⊕ ω_{odd} be the decomposition of ω into irreducible representations, and write ω_{even} − ω_{odd} for the difference in the Grothendieck group.

It follows from general principles that

ω_{even} − ω_{odd} = \frac{(−1)^{q}}{n!} ∑_{w∈W} w · π_{DS}.  (1.1)

Here q = \frac{1}{2}dim(S̃p(2n, ℝ)/K) = n(n + 1)/2, where K is a maximal compact subgroup. Also W denotes the complex Weyl group of type C_{n}, and π → w · π is the coherent continuation action of W on virtual modules.

The global character Θ_{DS} of π_{DS} is known explicitly as a function on the regular elements [5] [10]:

Θ_{DS} = ±\frac{∑_{w∈W_{K}} sgn(w)e^{wλ}}{∏_{α}(e^{α/2} − e^{−α/2})}  (1.2)

Here W_{K} is the Weyl group of K, λ is the Harish-Chandra parameter of π_{DS} and the product in the denominator is over the positive roots Δ^{+} defined by λ. This is Harish-Chandra’s formula on the compact Cartan subgroup, and appropriately interpreted holds on any Cartan subgroup by virtue of the fact that π_{DS} is a holomorphic representation.

Let θ_{even,odd} be the global character of ω_{even,odd}, as a function on the regular set. It follows immediately from (1.1), (1.2) and the definition of coherent continuation that

θ_{even} − θ_{odd} = \frac{± ∑_{w∈W} sgn(w)e^{wλ}}{∏_{α}(e^{α/2} − e^{−α/2})}  (1.3)

(the precise statement is Theorem 3.11). Furthermore λ = (n − \frac{1}{2}, n − \frac{3}{2}, …, \frac{1}{2}) and, formally at least, this is one-half the sum of the positive roots for the root

Typeset by AMS-TEX

Partially supported by NSF grant DMS-94-01074

system of type B_{n}. Therefore by the usual product formula the numerator of (1.3) equals

∏_{α∈Δ^{+}(B_{n})}(e^{α/2} − e^{−α/2}).  (1.4)

Write Δ^{+}(C_{n}) for the positive roots appearing in (1.2). The long roots of Δ^{+}(B_{n}) are the same as the short roots of Δ^{+}(C_{n}), whereas the short roots of Δ^{+}(B_{n}) are one-half times the long roots {α_{1}, …, α_{n}} of Δ^{+}(C_{n}). Thus

θ_{even} − θ_{odd} = \frac{± ∏_{Δ^{+}(B_{n})}(e^{α_{/}2} − e^{−α_{/}2})}{∏_{Δ^{+}(C_{n})}(e^{α_{/}2} − e^{−α_{/}2})}  (1.5)(a)
= \frac{±1}{∏_{j=1}^{n}(e^{α_{j}/4} + e^{−α_{j}/4})}.  (b)

Now suppose (for simplicity) that G is a connected complex reductive group, and g is a semisimple element of G. Let π be a finite dimensional representation of G such that for every weight α of π, −α is also a weight. Write the weights of π as a disjoint union S ∪ −S. Then

det(1 + π(g)) = ∏_{α∈S}(1 + e^{α}(g))(1 + e^{−α}(g))
= ∏_{α∈S}(e^{−α/2} + e^{α/2})(e^{α/2} + e^{−α/2})(g)  (1.6)

= ∏_{α∈S}(e^{α/2} + e^{−α/2})^{2}(g).

The weights of the standard representation of Sp(2n, ℝ) are ±α_{1}/2, …, ±α_{n}/2. This observation applied to (1.5)(b) gives

(θ_{even} − θ_{odd})(g̃)^{2} = \frac{1}{det(1 + g)}

where g is the image of g̃ in Sp(2n, ℝ). Therefore

(θ_{even} − θ_{odd})(g̃) = \frac{c(g̃)}{|det(1 + g)|^{\frac{1}{2}}}  (1.7)

with c(g̃)^{4} = 1.

With a little extra work, mostly having to do with the covering group, these formal arguments may be made precise, and the constants computed (Theorem 3.11 and Proposition 4.5).

We also describe an alternative formulation. We write a Cartan subgroup H̃ of S̃p(2n, ℝ) in terms of a certain cocycle, closely related to the Rao cocycle [11], so an element of H̃ is a pair (g; ϵ) with g ∈ H and ϵ = ±1. Then (1.7) takes the simple form (Proposition 5.11)

(θ_{even} − θ_{odd})(g; ϵ) = \frac{ϵ}{\sqrt{det(1 + g)}}.  (1.8)

Actually the oscillator representation depends on a choice of an additive character of ℝ, which enters on the right hand side of (1.8) in the choice of a branch of the square root.

Finally ω_{even} and ω_{odd} have distinct central characters. Evaluating (1.3) and (1.7) at an element zg̃, where z is a central element lying over the element −I of Sp(2n, ℝ) gives formulas (Propositions 3.18 and 4.7) for the global character of ω = ω_{even} ⊕ω_{odd}. However it is interesting to note the analogue of (1.8) has a more complicated numerator; similarly the analogue of (1.3) is less natural.

The character of the oscillator representation has been computed in ([14],§I, Théorème 2), using very different methods. Statement (1.7) (for ω_{even} ⊕ ω_{odd}) appears in [7] (without the computation of c(g̃)). This method of proof may be applied to compute characters of small representations of other groups.

In some sense (1.5)(a) is the main result of this paper. We write this as

θ_{even} − θ_{odd} = \frac{D_{SO}}{D_{Sp}}  (1.9)

where D_{Sp}, D_{SO} are Weyl denominators for Sp(2n, ℝ) and SO(2n + 1) respectively. Thus θ_{even} − θ_{odd} has the form of a transfer factor [9] between characters of S̃p(2n, ℝ) and SO(2n + 1). The formulas of this paper have been written with this application in mind, and we give a few details.

There is a bijection between stable, (strongly) regular semisimple conjugacy classes of SO(n+1, n) and Sp(2n, ℝ) – two such conjugacy classes correspond if they have the same non-trivial eigenvalues. Equivalently if H is a Cartan subgroup of SO(n+1, n) we may choose an isomorphism ϕ : H → H′ with a Cartan subgroup of Sp(2n, ℝ). Let Θ be a stable invariant eigendistribution on SO(n + 1, n), identified with a function on the (strongly) regular semisimple elements.

We define a function Θ′ on S̃p(2n, ℝ) by

Θ′(g′) = Θ(g)(θ_{even} − θ_{odd})(g′) (g′ ∈ S̃p(2n, ℝ))

where g corresponds to the image p(g′) ∈ Sp(2n, ℝ) of g′. This is independent of the choice of g since Θ is stable.

Roughly speaking, using the identification of H and H′, a stable invariant eigendistribution for SO(n + 1, n), when multiplied by θ_{even} − θ_{odd}, is a candidate for an invariant eigendistribution on S̃p(2n, ℝ). This follows from Harish-Chandra’s formula for the restriction of an invariant eigendistribution to a Cartan subgroup [4]: Θ and Θ′ have the same “numerator” on H ≃ H′. This can be made precise, and the map Θ → Θ′ is a bijection between stable invariant eigendistributions on SO(n + 1, n) and genuine, stable invariant eigendistributions on S̃p(2n, ℝ). This application is discussed [1].

§2. Coherent Continuation.

We fix once and for all a real vector space W of dimension 2n equipped with a non-degenerate symplectic form <, >, and we fix a standard basis e_{1}, …, f_{n} of W. The isometry group of <, > is then Sp(2n, ℝ) realized as the set of 2n × 2n real

matrices satisfying gJ ^{t}g^{−1} = J, where J = ( \begin{matrix}& I_{n} \\ −I_{n} \end{matrix} ). The real and complex Lie algebras are denoted 𝔰𝔭(2n, ℝ) and 𝔰𝔭(2n, ℂ) respectively, realized as real or complex matrices X satisfying XJ + J ^{t}X = 0. Let S̃p(2n, ℝ) be the two-fold cover of Sp(2n, ℝ), with covering map p : S̃p(2n, ℝ) → Sp(2n, ℝ). It is well known that S̃p(2n, ℝ) is unique up to isomorphism. In §5 we will choose a particular model of S̃p(2n, ℝ), but until then this will not be necessary.

Let K̃ be the inverse image in S̃p(2n, ℝ) of the usual maximal compact subgroup K = Sp(2n, ℝ) ∩ SO(2n, ℝ) ≃ U(n) of Sp(2n, ℝ). Unless otherwise noted by “module” we mean admissible (𝔰𝔭(2n, ℂ), K̃)-module, and by virtual module we mean an element of the corresponding Grothendieck group, i.e. a formal finite linear combination of irreducible modules with integral coefficents.

Given a non-trivial unitary additive character ψ of ℝ the associated oscillator representation ω(ψ) is the direct sum ω(ψ) = ω(ψ)_{even} ⊕ ω(ψ)_{odd}, where ω(ψ)_{even} and ω(ψ)_{odd} are irreducible unitary representations and ω(ψ)_{even} contains a K̃- invariant line. (In the usual model ω(ψ)_{even,odd} is realized on a space of even (odd) functions, respectively.) According to the Hermitian symmetric structure write 𝔰𝔭(2n, ℂ) = 𝔨 ⊕ 𝔭 = 𝔨 ⊕ 𝔭^{+} ⊕ 𝔭^{−} as usual. Define ψ_{±}(x) = e^{±2πix}. We label 𝔭^{±} so that ω(ψ_{±})_{even,odd} have cyclic vectors under the action of 𝔭^{±}, equivalently a vector annihlated by 𝔭^{∓}. We reserve the term “highest weight module” for a module with a vector annihlated by 𝔭^{−}.

For a_{1}, …, a_{n} ∈ ℂ let (a_{1}, …, a_{n}) = ( \begin{matrix}& X \\ −X \end{matrix} ) with X = diag(a_{1}, …, a_{n}).

The set of such elements with a_{i} ∈ ℝ (resp. ℂ) forms the real (resp. complexified) Lie algebra 𝔱_{0} (resp. 𝔱) of a compact Cartan subgroup T of Sp(2n, ℝ). For H a subgroup of Sp(2n, ℝ) we let H̃ denote its inverse image in S̃p(2n, ℝ). The compact Cartan subgroup T̃ of S̃p(2n, ℝ) is connected and abelian (cf. §3). We write (a_{1}, …, a_{n}) for the element of 𝔱^{∗} = Hom(𝔱, ℂ) taking (z_{1}, …, z_{n}) to i ∑ a_{j}z_{j}. Then the set of roots of 𝔱 in 𝔰𝔭(2n, ℂ) is {±e_{i} ± e_{j}, ±2e_{i}} as usual. The weight lattice of 𝔱 in 𝔰𝔭(2n, ℂ) is Λ = {(a_{1}, …, a_{n}) | a_{i} ∈ ℤ}. Let Δ_{c}^{+} = {e_{i} − e_{j}|i < j} and Δ_{n}^{+} = {e_{i}, e_{i} + e_{j}}, the positive compact and non-compact roots, and let Δ^{+} = Δ_{n}^{+} ∪ Δ_{c}^{+}. Write ρ_{c}, ρ_{n} and ρ for one-half the sum of the positive roots in these sets respectively. The finite-dimensional irreducible representations of K̃ are parametrized by highest weights (a_{1}, …, a_{n}) with a_{1} ≥ a_{2} ≥ ⋯ ≥ a_{n}, a_{i} ∈ ℤ for all i or a_{i} ∈ ℤ + \frac{1}{2} for all i.

The infinitesimal character of ω(ψ) is λ_{0} = (n − \frac{1}{2}, n − \frac{3}{2}, …, \frac{1}{2}) (identified via the Harish-Chandra homomorphism). We work entirely in the category of representations with this infinitesimal character.

Definition 2.1. Let π be a virtual module with infinitesimal character λ_{0}. Let Φ be a coherent family based on T̃ and

λ_{0} + Λ = \{(a_{1}, …, a_{n}) | a_{i} ∈ ℤ + \frac{1}{2}\}

([15], Definition 7.2.5 and Corollary 7.2.27). For w ∈ W = W (𝔰𝔭(2n, ℂ), 𝔱) define

w · π = Φ(w^{−1}λ_{0})

([15], Definition 7.2.28). This is a virtual module with infinitesimal character λ_{0}.

A few comments are in order since we are not precisely in the setting of [15]. Since T̃ is connected we freely identify the formal symbols λ_{0} +Λ of ([15], Definition 7.2.5) with elements of 𝔱^{∗}. Although S̃p(2n, ℝ) is not a linear group, ([15], Corollary 7.2.27) carries over without any changes. The integral roots Δ(λ_{0}) and integral Weyl group W (λ_{0}) of λ_{0} are of type D_{n}; however ([15], Lemma 7.2.29) is easily seen to hold for the larger set {w ∈ W | wλ_{0} − λ is a sum of weights} which in our case is all of W.

We note that the more detailed results on wall-crossing of ([15], §7) fail with W in place of W (λ_{0}) (cf. Remark 2.10).

Now let π_{DS} be the holomorphic discrete series representation of S̃p(2n, ℝ) with infinitesimal character λ_{0}. This has lowest K-type (n + \frac{1}{2}, …, n + \frac{1}{2}), and is a highest weight module. Let

π̂_{DS} = ∑_{w∈W} sgn(w)w · π_{DS}.  (2.2)

Theorem 2.3.

π̂_{DS} = (−1)^{q}n![ω(ψ_{+})_{even} − ω(ψ_{+})_{odd}].

Proof. It follows immediately from the definition that all simple roots α of Δ^{+}(λ_{0}) = Δ^{+} ∩ Δ(λ_{0}) are in the τ-invariant [15] of π̂_{DS}. Therefore by ([16], Corollary 4.7) any irreducible summand σ of π̂_{DS} has Gelfand-Kirillov dimension n. Since the irreducible summands of a highest weight module tensored with a finite dimensional representation are all highest weight modules, this holds also for σ. By the classification of highest weight modules for S̃p(2n, ℝ) [3], the only possibilites for σ are ω(ψ_{+})_{even} and ω(ψ_{+})_{odd}. Thus

π̂_{DS} = aω(ψ_{+})_{even} + bω(ψ_{+})_{odd}

for some integers a, b. To evaluate a, b we use the following Blattner-type formula.^{1} Let m(μ, π̂_{DS}) denote the multiplicity of a K-type μ in π̂_{DS}.

Proposition 2.4. Let μ be an irreducible representation of K̃ (identified with its highest weight). Let Q denote the Kostant multiplicity function for the set Δ_{n}^{+} [8]. Then

m(μ, π̂_{DS}) = ∑_{w∈W} ∑_{y∈W_{K}} sgn(w)sgn(y)Q(y(μ + ρ_{c}) − (wλ_{0} + ρ_{n})).

This is well-known to the experts, but for lack of a reference we give a proof.

Proof. We prove the stronger statement

m(μ, w · π_{DS}) = ∑_{y∈W_{K}} sgn(y)Q(y(μ + ρ_{c}) − (w^{−1}λ_{0} + ρ_{n})).  (2.5)

1We thank the referee for pointing out that Theorem 2.3 may also be proved by considering π̂_{DS} as a Verma module, and applying standard results in this category.

Write π_{DS} as a derived functor module R_{𝔮}^{S}(λ_{0}), also satisfying R_{𝔮}^{i}(λ_{0}) = 0 for i ≠ S ([17], Theorem 6.8). Here 𝔮 = 𝔱 ⊕ 𝔲 is the Borel subalgebra defined by Δ^{+} (i.e. making λ_{0} dominant), and S = \frac{1}{2}dim(K/T) = n(n − 1)/2.

Thus

m(μ, w · π_{DS}) = m(μ, w · R_{𝔮}^{S}(λ_{0}))

= m(μ, w · ∑_{i}(−1)^{S−i}R_{𝔮}^{i}(λ_{0})).

By ([15], Corollary 7.2.10), this equals

m(μ, ∑_{i}(−1)^{S−i}R_{𝔮}^{i}(w^{−1}λ_{0})).

Now (2.5) follows from a Blattner formula ([15], Theorem 6.3.12); see also ([18], Theorem 6.5.3).

It is not difficult to evaluate the sum of Proposition 2.4 for μ_{even} = (\frac{1}{2}, …, \frac{1}{2}) and μ_{odd} = (\frac{3}{2}, \frac{1}{2}, …, \frac{1}{2}), the lowest K-types of ω(ψ_{+})_{even,odd} respectively. Since Δ_{n}^{+} is invariant under W_{K}, yρ_{n} = ρ_{n} and Q(yτ) = Q(τ) for all y ∈ W_{K} and all τ. Replacing w by y^{−1}λ and changing variables shows

m(μ, π̂_{DS}) = n! ∑_{w∈W} sgn(w)Q(μ + ρ_{c} − ρ_{n} − wλ_{0}).  (2.6)

Define w_{0} by: w_{0}ρ_{c} = ρ_{c} and w_{0}ρ_{n} = −ρ_{n}; the length of w_{0} is the number of positive non-compact roots, i.e. q = n(n + 1)/2. Let τ = μ_{even} + ρ_{c} − ρ_{n} = (−\frac{1}{2}, −\frac{3}{2}, …, −n + \frac{1}{2}). Then w_{0}λ_{0} = τ, and replacing w by ww_{0} gives

m(μ_{even}, π̂_{DS}) = n!(−1)^{q} ∑_{w∈W} sgn(w)Q(τ − wτ).  (2.7)

It is easy to see Q(τ − wτ) is non-zero if and only if w = 1, giving m(μ_{even}, π̂_{DS}) = n!(−1)^{q}.

The computation for μ_{odd} is similar, with τ replaced by τ + (1, 0, …, 0), and w_{0} replaced by s_{2e_{1}}w_{0}. It follows that m(μ_{odd}, π̂_{DS}) = −n!(−1)^{q}. This completes the proof of Theorem 2.3.

The Weyl group W (D_{n}) of type D_{n} is embedded naturally in W. Let ϕ : W → ±1 be the corresponding quotient map:

1 → W (D_{n}) → W →^{ϕ} ℤ/2ℤ → 1  (2.8)(a)

For later use we note that ϕ(w) = sgn(w)sgn(p(w)) where p : W → W_{K} is given by

1 → (ℤ/2ℤ)^{n} → W →^{p} W_{K} ≃ S^{n} → 1.  (2.8)(b)

The next result is a corollary of the proof of Theorem 2.3.

Corollary 2.9.

∑_{ϕ(w)=(−1)^{n}} sgn(w)w · π_{DS} = n!(−1)^{q}ω(ψ_{+})_{even}
∑_{ϕ(w)=(−1)^{n+1}} sgn(w)w · π_{DS} = −n!(−1)^{q}ω(ψ_{+})_{odd}

For any σ with ϕ(σ) = −1,

σ · ω(ψ_{+})_{even} = ω(ψ_{+})_{odd}.

Dualizing, the corresponding result holds for the antiholomorphic discrete series representation π_{DS}^{∗} and ω(ψ_{−})_{even}, ω(ψ_{−})_{odd}.

Remark 2.10. The root 2e_{n} is simple for Δ^{+}, but is not in the integral root system Δ(λ_{0}). Neverthless the action of σ = s_{2e_{n}} is defined. This is an example of “exotic” wall-crossing (though not of an integral wall). In particular σ takes the irreducible representation ω(ψ_{+})_{even} to an irreducible representation (ω(ψ_{+})_{odd}), something which is forbidden for ordinary (integral) wall crossing ([15], Corollary 7.3.19).

Remark 2.11. Let π̂′_{DS} = ∑_{w} sgn(w)w ·π′_{DS} where π′_{DS} is any discrete series representation with infinitesimal character λ_{0}. By Harish-Chandra’s character formula the characters of π̂_{DS} and π̂′_{DS} have the same restriction to the elliptic set, and it follows that

π̂′_{DS} = a[ω(ψ_{+})_{even} − ω(ψ_{+})_{odd}] + b[ω(ψ_{−})_{even} − ω(ψ_{−})_{odd}]

for some integers a+b = (−1)^{q}n!. However the evaluation of a, b by means of Proposition 2.4 is not at all easy outside of the holomorphic/anti-holomorphic cases. For example for Sp(6, ℝ) and π′_{DS} a large discrete series representation (i.e. corresponding to a Weyl chamber with no simple compact roots), a, b = 4, 2 or 2, 4.

§3. Character Formulas.

We begin by realizing representatives for the conjugacy classes of Cartan subgroups via Cayley transforms based on T as in [12].

For S a set of strongly orthogonal non-compact roots of 𝔱, let c = c^{S} be a Cayley transform associated to S. This is an element of Sp(2n, ℂ), and 𝔥 = ad(c)(𝔱) is a Cartan subalgebra of 𝔰𝔭(2n, ℂ). Let 𝔥_{0} = 𝔥 ∩ 𝔰𝔭(2n, ℝ). The centralizer of 𝔥_{0} in Sp(2n, ℝ) is a Cartan subgroup of Sp(2n, ℝ). The inverse image H̃ of H in S̃p(2n, ℝ) is a Cartan subgroup of S̃p(2n, ℝ); H̃ is abelian and equal to the centralizer of 𝔥_{0} in S̃p(2n, ℝ). Every Cartan subgroup of Sp(2n, ℝ) or S̃p(2n, ℝ) is conjugate to one obtained in this manner.

We now write Δ^{+}(T) for the positive roots Δ^{+} defined in §1. Then Δ^{+}(H) = c^{∗−1}Δ^{+}(T) is a set of positive roots of 𝔥 (c^{∗} : 𝔥^{∗} → 𝔱^{∗} is the adjoint of c). Then Δ^{+}(H) comes equipped with a set of strongly orthogonal real roots which we write

β_{1}, …, β_{m}, γ_{1}, …, γ_{s}  (3.1)

where β_{1}, …, β_{m} are short and γ_{1}, …, γ_{s} are long.

As is well known K̃ is isomorphic to the cover of K ≃ U(n) defined by the square root of the determinant [2]. This is the group {(g, z) | g ∈ U(n), z ∈ ℂ^{∗}, det(g) = z^{2}}. It follows that the compact Cartan subgroup T̃ of S̃p(2n, ℝ) is connected and the exponential map ẽxp : 𝔱_{0} → T̃ is surjective with kernel

{2π(a_{1}, …, a_{n}) ∈ 𝔱_{0}|a_{i} ∈ ℤ, ∑_{i} a_{i} ∈ 2ℤ}.  (3.2)

For λ ∈ λ_{0} + Λ (Definition 2.1) and t = ẽxp(X) ∈ T̃, define e^{λ}(t) = e^{λ(X)} as usual.

For a Cayley transform c and H as above we write H = T_{H}A with T_{H} = T ∩ H and A = exp(𝔞_{0} = 𝔥_{0} ∩ 𝔭_{0}). (We identify the exponential of 𝔞_{0} in S̃p(2n, ℝ) with exp(𝔞_{0}) ∈ Sp(2n, ℝ)). Then H̃ ≃ T̃_{H}A, and we may write elements of H̃ in the form

g = ẽxp(Z)exp(X) (Z ∈ 𝔱_{0}, X ∈ 𝔞_{0}).  (3.3)(a)

For λ ∈ λ_{0} + Λ ⊂ 𝔱^{∗} and g ∈ H̃ define

e^{λ}(g) = e^{λ}(t)e^{c^{∗−1}λ(X)}  (b)

where g = t · exp(X) ∈ T̃_{H}A. Equivalently for λ ∈ 𝔥^{∗}, c^{∗}(λ) ∈ λ_{0} + Λ define

e^{λ}(g) = e^{c^{∗}λ}(t)e^{λ(X)}.  (c)

Proposition 3.4. Let c be a Cayley transform with corresponding Cartan subgroup H̃ of S̃p(2n, ℝ). Write γ_{1}, …, γ_{s} for the long real roots of Δ^{+}(H). Let g be a regular element of H̃ satisfying

|e^{γ_{i}}(g)| < 1 i = 1, …, s.  (3.5)

Then

[θ(ψ_{+})_{even} − θ(ψ_{+})_{odd}](g) = \frac{∑_{w∈W} sgn(w)e^{wλ_{0}}}{∏_{α∈Δ^{+}(H)}(e^{α/2} − e^{−α/2})}(g).  (3.6)

Every regular element of S̃p(2n, ℝ) is conjugate to some g satisfying (3.5), so (3.6) determines θ(ψ_{+})_{even}−θ(ψ_{+})_{odd} completely. This is made explicit in Theorem 3.11. Note the denominator factors to Sp(2n, ℝ).

Proof. Let Θ_{DS} (resp. Θ̂_{DS}) the character of π_{DS} (resp. π̂_{DS}). By Theorem 2.3 (3.6) is equivalent to

Θ̂_{DS}(g) = n!(−1)^{q} \frac{∑_{w∈W} sgn(w)e^{wλ_{0}}}{∏_{α∈Δ^{+}(H)}(e^{α/2} − e^{−α/2})}(g).  (3.7)

This follows more or less immediately from [12], [19], and the definition of coherent continuation.

We first show for g in the smaller set (cf. 3.1):

|e^{β_{i}}(g)| < 1 (i = 1, …, m),  (3.8)

Θ_{DS}(g) = (−1)^{q} \frac{∑_{W_{K}} sgn(w)e^{wλ_{0}}}{∏_{α∈Δ^{+}(H)}(e^{α/2} − e^{−α/2})}(g).  (3.9)

By [12] this holds with λ_{0} replaced by ρ = λ_{0} + (\frac{1}{2}, …, \frac{1}{2}), and Θ_{DS} replaced by the holomorphic discrete series representation with infinitesimal character ρ. Then (3.9) follows by deforming ρ to λ as in ([19], Corolloary 3.8). By the version of coherent continuation of [6] this implies (3.7), still for g in the smaller set (3.8).

Now suppose |e^{β_{i}}(g)| > 1 for some i. Let s_{i} = s_{β_{i}} ∈ W (𝔰𝔭, 𝔥). Also write s_{i} for the corresponding reflection in W (𝔰𝔭, 𝔱), i.e. reflection in the root c^{∗}β_{i} ∈ 𝔱^{∗}. Write g = ẽxp(Z)exp(X) as in (3.3)(a). It follows from a calculation in S̃p(2n, ℝ) that g is conjugate via S̃p(2n, ℝ) to g′ = ẽxp(s_{i}Z)exp(s_{i}X). This calculation is entirely in the covering group of GL(n, ℝ), on which the cocycle has the simple form ([11], Corollary 5.5(2)). The right hand side of (3.7) is invariant upon replacing g by g′. It follows that (3.7) holds without condition (3.8), and this completes the proof.

The discussion at the end of the proof can be extended to consider reflections s_{i} in the long roots γ_{i}. It is easy to see that ẽxp(Z) exp(X) (Z ∈ 𝔱_{0}, X ∈ 𝔞_{0}) is conjugate via S̃p(2n, ℝ) to ẽxp(Z)exp(s_{i}X), rather than ẽxp(s_{i}Z)exp(s_{i}X). This reduces to the split Cartan subgroup H̃ = T̃_{H}A of S̃L(2, ℝ), and follows from the fact that T̃_{H} ≃ ℤ/4ℤ is the center of S̃L(2, ℝ). Furthermore for any τ ∈ λ_{0} + Λ

e^{τ} (ẽxp(Z))/e^{τ} (ẽxp(s_{i}Z)) = e^{γ_{i}/2}(ẽxp(Z)) = ±1.  (3.10)

We state the resulting formula, at the same time treating the case of ω(ψ_{−}).

Theorem 3.11.

[θ(ψ_{±})_{even}−θ(ψ_{±})_{odd}](g)

= ∏_{i=1}^{s} sgn(1 + e^{±γ_{i}/2}(g))\frac{∑_{w∈W} sgn(w)e^{wλ_{0}}}{∏_{α∈Δ^{+}(H)}(e^{α/2} − e^{−α/2})}(g)

= ∏_{i=1}^{s} sgn(1 + e^{±γ_{i}/2}(g))\frac{1}{∏_{i=1}^{n}(e^{α_{i}/4} + e^{−α_{i}/4})(g)}.

Proof. The first line for ψ_{+} follows from the preceding discussion. With ^{∗} denoting contragredient, θ(ψ_{−})(g) = θ(ψ_{+})^{∗}(g) = θ(ψ_{+})(g^{−1}). The quotient \begin{matrix}∑ & _{w} \\ ∏ & _{α} \end{matrix} is invariant under g → g^{−1}, and e^{γ_{i}/2}(g^{−1}) = e^{−γ_{i}/2}(g). The second line follows exactly as in the introduction (cf. 1.5); there is no problem making this argument precise.

Remark 3.12. The terms sgn(1+e^{±γ_{i}/2}(g)) are constant on each connected component of a Cartan subgroup, and on the identity component of each Cartan subgroup are independent of ψ_{±}. In particular the restriction of the θ(ψ)_{even} − θ(ψ)_{odd} to a neighborhood of the identity in S̃p(2n, ℝ) is independent of ψ.

We now make explicit choices of Cayley transforms and Cartan subgroups. Our discussion, as well as some notation, is similar to but not exactly the same as [14].

For non-negative integers m, r, s with 2m+r+s = n we define a Cartan subgroup H^{m,r,s} of Sp(2n, ℝ) with Lie algebra 𝔥_{0}^{m,r,s}, and a Cayley transform c : 𝔱 ≃ 𝔥^{m,r,s}. Write W = ℝ^{2n} = W_{1} ⊕ W_{2} ⊕ W_{3} where W_{1} is spanned by {e_{i}, f_{i}|1 ≤ i ≤ 2m}, W_{2} by {e_{i}, f_{i}|2m + 1 ≤ i < 2m + r} and W_{3} by {e_{i}, f_{j}|2m + r + 1 ≤ i ≤ n}. We identify Sp(W_{i}) and 𝔰𝔭(W_{i}) with their images in Sp(2n, ℝ) and 𝔰𝔭(2n, ℝ). For z_{i} = x_{i} + iy_{i} ∈ ℂ, 1 ≤ i ≤ m let

𝔥^{m,0,0}(z_{1}, …, z_{m}) = ( \begin{matrix}& X & Y \\ X & & & −Y \\ −Y & & & −X \\ & Y & −X \end{matrix} ) ∈ 𝔰𝔭(W_{1})  (3.13)(a)

where X = diag(x_{1}, …, x_{m}) and Y = diag(y_{1}, …, y_{m}). For θ_{i} ∈ ℝ (1 ≤ i ≤ r) we let

𝔥^{0,r,0}(θ_{1}, …, θ_{r}) = ( \begin{matrix}& X \\ −X \end{matrix} ) ∈ 𝔰𝔭(W_{2})  (b)

with X = diag(θ_{1}, …, θ_{r}), and for x_{i} ∈ ℝ (1 ≤ i ≤ s) let

𝔥^{0,0,s}(x_{1}, …, x_{s}) = diag(x_{1}, …, x_{s}, −x_{1}, …, −x_{s}) ∈ 𝔰𝔭(W_{3}).  (c)

Taking the sum of these elements gives us an element

𝔥^{m,r,s}(z_{1}, …, z_{m}, θ_{1}, …, θ_{r}, x_{1}, …, x_{s}) ∈ 𝔰𝔭(2n, ℝ)  (d)

and this defines a Cartan subalgebra 𝔥_{0}^{m,r,s} of 𝔰𝔭(2n, ℝ). The compact Cartan subalgebra is 𝔱_{0} = 𝔥_{0}^{0,n,0}. The complexification 𝔥^{m,r,s} is defined in the obvious way.

Let H^{m,r,s} ≃ ℂ^{∗m} × S^{1r} × R^{∗s} be the Cartan subgroup of Sp(2n, ℝ) with Lie algebra 𝔥_{0}^{m,r,s}, with inverse image H̃^{m,r,s} in S̃p(2n, ℝ). These are representatives for the conjugacy classes of Cartan subgroups of Sp(2n, ℝ) and S̃p(2n, ℝ) respectively. The compact Cartan subgroup is T = H^{0,n,0}.

Given H^{m,r,s} let

c_{1} = \frac{\sqrt{2}}{2} ( \begin{matrix}I_{2m} & −iI′_{2m} \\ −iI′_{2m} & I_{2m} \end{matrix} )

where I′_{2m} = antidiag(1, …, 1), let c_{2} = \frac{\sqrt{2}}{2} ( \begin{matrix}I_{r} & −iI_{r} \\ −iI_{r} & I_{r} \end{matrix} ) and let c = c_{1} × c_{2} ×

I_{s} ∈ Sp(W_{1ℂ}) × Sp(W_{2ℂ}) × Sp(W_{3}) ⊂ Sp(2n, ℂ). Then c is a Cayley transform for H^{m,r,s}, i.e. ad(c)(𝔱) = 𝔥^{m,r,s}.

Recall Δ^{+}(H^{m,r,s}) = c^{∗−1}(Δ^{+}(T)). For i = 1, …, n let α_{i} = c^{∗−1}(2e_{i}); these are the long roots of Δ^{+}(H^{m,r,s}). The strongly orthogonal real roots β_{i}, γ_{i} (cf. 3.1) are β_{i} = c^{−1}(e_{2i−1} + e_{2i}) (i = 1, … m) and γ_{i} = c^{−1}(2e_{2m+r+i}) (i = 1, …, s). For later use we compute the Weyl reflections corresponding to β_{i}, γ_{i}. Write X as in (3.13)(d). Then β_{i}(X) = z_{i} + \overline{z}_{i} and γ_{i}(X) = 2x_{i}. Therefore

s_{β_{i}}(X) = 𝔥^{m,r,s}(z_{1}, …, −\overline{z}_{i}, …, z_{m}, θ_{1}, …, θ_{r}, x_{1}, …, x_{s})  (3.14)(a)

and

s_{γ_{i}}(X) = 𝔥^{m,r,s}(z_{1}, …, z_{m}, θ_{1}, …, θ_{r}, x_{1}, …, −x_{i}, … x_{s}).  (b)

We make the decomposition (3.3)(a) more explicit. Write g ∈ H̃^{m,r,s} as

g = t · exp(X) = ẽxp(Z)exp(X).  (3.15)(a)

Here

X = 𝔥_{0}^{m,r,s}(x_{1}, …, x_{m}, \overbrace{0, …, 0}^{r}, c_{1}, …, c_{s}) ∈ 𝔞_{0} (x_{i}, c_{j} ∈ ℝ)  (b)

and

Z = 𝔥^{0,n,0}(y_{1}, −y_{1}, …, y_{m}, −y_{m}, θ_{1}, …, θ_{r}, πk_{1}, …, πk_{s}) +

\frac{1 − δ}{2} 𝔥^{0,n,0}(2π, 0, …, 0) (c)

for some y_{i}, θ_{j} ∈ ℝ, k_{i} ∈ ℤ and δ = ±1. The parameter δ has been chosen for later convenience (cf. Proposition 4.5).

Remark 3.16. The choice of Z is not at all unique. If r+s ≠ 0 we may assume δ = 1 (by modifying the first term of Z if necessary). Only in the case 2m = n, r = s = 0 is it necessary to take δ = −1. In this case we can replace this term with the more natural

Z′ = 𝔥^{0,n,0}(π, …, π, (−1)^{m+1}π);

note that ẽxp(Z′) is a central element of S̃p(2n, ℝ) lying over −I ∈ Sp(2n, ℝ). In these coordinates

sgn(1 + e^{±γ_{i}}(g)) = (∓sgn(c_{i}))^{k_{i}}  (3.17)

(cf. Theorem 3.11).

We compute the character of the oscillator representation in these coordinates. Recall p(w) ∈ W_{K} was defined in (2.8)(b). This also follows from ([14], §1, 7.8).

Proposition 3.18. Write g = t · exp(X) as in (3.15).

[θ(ψ_{±})_{even}+θ(ψ_{±})_{odd}](g)

= (∓1)^{n} ∏_{i=1}^{s}(∓sgn(c_{i}))^{k_{i}+1} \frac{∑_{w∈W} sgn(p(w))e^{wλ_{0}}}{∏_{α∈Δ^{+}(H^{m,r,s})}(e^{α/2} − e^{−α/2})}(g)

= (∓1)^{n} ∏_{i=1}^{s}(∓sgn(c_{i}))^{k_{i}+1} \frac{1}{∏_{i=1}^{n}(e^{α_{i}/4} − e^{−α_{i}/4})(g)}

Proof. Let Z_{0} = 𝔥^{0,n,0}(π, …, π) and let z = ẽxp(Z_{0}). This is a central element of S̃p(2n, ℝ) lying over −I ∈ Sp(2n, ℝ). Then θ(ψ_{±})_{even}(z) = i^{±n}, as is seen by considering the lowest K-type ±(\frac{1}{2}, …, \frac{1}{2}) of θ(ψ_{±})_{even}. On the other hand θ(ψ_{±})_{odd}(z) = −i^{±n}. Therefore

[θ(ψ_{±})_{even} + θ(ψ_{±})_{odd}](g) = i^{∓n}[θ(ψ_{±})_{even} − θ(ψ_{±})_{odd}](zg).  (3.19)

We compute the right hand side by Theorem 3.11. Replacing g by zg replaces Z (3.15)(c) by Z +Z_{0}, hence k_{i} is replaced by k_{i} +1. This accounts for the (∓sgn(c_{i})) term. Each term e^{wλ_{0}}(g) is multiplied by e^{wλ_{0}}(Z_{0}). This is readily computed to be i^{n^{2}}(−1)^{t(w)} where t(w) is the number of sign changes in w. Note that (−1)^{t(w)} = sgn(w)sgn(p(w)) with p as in (2.8)(b). Finally each non-compact root term in the Weyl denominator changes sign, giving a factor of (−1)^{q}. The constant is therefore i^{∓n}i^{n^{2}}(−1)^{q} = (∓1)^{n}. This proves the first statement. The second follows from the fact, left as an exercise, that the numerator equals ∏_{α}(e^{α/2} − e^{−α/2}) ∏_{i}(e^{α_{i}/4} + e^{−α_{i}/4}), where the first product is over the short positive roots. This completes the proof.

§4. Determinants.

We state as lemmas some straightforward calculations on the compact and split Cartan subgroups of SL(2, ℝ), and the complex Cartan subgroup of Sp(4, ℝ). The proofs are left to the reader.

Definition 4.1. For θ ∈ ℝ let

ζ(θ) = \{ \begin{matrix}sgn cos(θ) & cos(θ) ≠ 0, \\ sin(θ) & cos(θ) = 0. \end{matrix}

Lemma 4.2. For θ ∈ ℝ let X_{θ} = 𝔥^{0,1,0}(θ) = ( \begin{matrix}0 & θ \\ −θ & 0 \end{matrix} ), an element of the compact

Cartan subalgebra 𝔱_{0} of 𝔰𝔩(2, ℝ), and let t̃_{θ} = ẽxp(X_{θ}) ∈ S̃L(2, ℝ). For α a root of 𝔱,

\frac{1}{e^{α/4} + e^{−α/4}} (t̃_{θ}) = \frac{1}{2 cos(θ/2)}.  (a)

Let t_{θ} = p(t̃_{θ}) ∈ SL(2, ℝ). Then

det(1 + t_{θ}) = 4 cos^{2}(θ/2) ≥ 0  (b)

and

\sqrt{det(1 + t_{θ})} = |det(1 + t_{θ})|^{\frac{1}{2}} = 2 cos(θ/2)ζ(θ/2).  (c)

Lemma 4.3. For c_{0} ∈ ℝ let X = 𝔥^{0,0,1}(c_{0}) = diag(c_{0}, −c_{0}), an element of the split Cartan subalgebra 𝔞_{0} of 𝔰𝔩(2, ℝ). For k ∈ ℤ let Z = 𝔥^{0,1,0}(πk) ∈ 𝔱_{0}, and write c = c_{0} + πik. Let ã_{c} = ẽxp(Z)exp(X) ∈ Ã = H̃^{0,0,1} ⊂ S̃L(2, ℝ).

For α a root of 𝔞,

\frac{1}{e^{α/4} + e^{−α/4}} (ã_{c}) = \frac{1}{2 cosh(c/2)}  (a)

Let a_{c} = p(ã_{c}) = diag(e^{c}, e^{−c}) ∈ SL(2, ℝ). Then

det(1 + a_{c}) = (2 cosh(c/2))^{2}  (b)

and

|det(1 + a_{c})|^{\frac{1}{2}} = i^{k}(−sgn(c_{0}))^{k}2 cosh(c/2).  (c)

Lemma 4.4.

For z ∈ ℂ let Z = 𝔥^{1,0,0}(z) ∈ 𝔰𝔭(4, ℝ). For δ = ±1 let Z_{0} be the element \frac{1−δ}{2} 𝔥^{0,2,0}(π, π) of the compact Cartan subalgebra 𝔱_{0} of 𝔰𝔭(4, ℝ) (cf. Remark 3.16). Let g̃ = ẽxp(Z_{0})ẽxp(Z) ∈ S̃p(4, ℝ). With ±β_{1}, ±β_{2} be the long roots of 𝔥^{1,0,0},

\frac{1}{(e^{β_{1}/4} + e^{−β_{1}/4})(e^{β_{2}/4} + e^{−β_{2}/4})}(g̃) = \frac{δ}{2(cosh(x) + δ cos(y))}.  (a)

Let g = p(g̃) ∈ Sp(4, ℝ). Then

det(1 + g) = [2(cosh(x) + δ cos(y))]^{2} ≥ 0  (b)

and

\sqrt{det(1 + g)} = |det(1 + g)|^{\frac{1}{2}} = 2(cosh(x) + δ cos(y))  (c)

Proposition 4.5. With notation as in Corollary 3.18,

[θ(ψ_{±})_{even} − θ(ψ_{±})_{odd}](g̃) = \frac{∏_{i=1}^{p} ζ(θ_{i}/2) ∏_{j=1}^{s}(±i)^{k_{j}}δ}{|det(1 + g)|^{\frac{1}{2}}} .

Proof. This follows immediately from Theorem 3.11 and Lemmas 4.2-4.

Recall (Remark 3.16) we may assume δ = 1 except in the case 2m = n, r = s = 0. For the definition and properties of the Weil invariant γ(x, ψ) we refer to ([11],

Appendix). For x ∈ ℝ^{∗} let \sqrt[ψ]{x} = γ(x, ψ)\sqrt{|x|}. This is a choice of square root of x and satisfies \sqrt[ψ]{xy} = \sqrt[ψ]{x} \sqrt[ψ]{y}(x, y)_{ℝ} where (,)_{ℝ} is the Hilbert symbol [13]. Also γ(x, ψ_{a}) = γ(x, ψ_{ab^{2}}) for all b ∈ ℝ^{∗}, and γ(x, ψ_{±}) = 1 (x > 0) or ∓i (x < 0).

We express the preceding result in terms of \sqrt[ψ]{det(1 + g)}. In Lemmas 4.2 and 4.4, \sqrt[ψ]{det(1 + g)} = |det(1 + g)|^{\frac{1}{2}} , whereas in Lemma 4.3 a short calculation gives

\sqrt[ψ]{det(1 + g)} = γ((−1)^{k}, ψ)| det(1 + g)|^{\frac{1}{2}} .

Noting that γ((−1)^{k}, ψ_{±}) = (∓i)^{k}(−1)^{k(k−1)/2} gives the next result.

Proposition 4.6. Notation as in Proposition 3.18.

[θ(ψ)_{even} − θ(ψ)_{odd}](g̃) = \frac{∏_{i=1}^{p} ζ(θ_{i}/2) ∏_{j=1}^{s}(−1)^{k_{j}(k_{j}−1)/2}δ}{\sqrt[ψ]{det(1 + g)}}

Note that the numerator is independent of ψ, and the role of ψ is solely in the branch of the square root. Finally we proceed as in the proof of Proposition 3.18 to obtain a result for θ(ψ_{±})_{even} + θ(ψ_{±})_{odd} (cf. [14], §I, Théorème 2).

Proposition 4.7. Notation as in Proposition 3.18.

[θ(ψ_{±})_{even} + θ(ψ_{±})_{odd}](g̃) = (±i)^{p} \frac{∏_{i=1}^{p} sgn(sin(θ_{i}/2)) ∏_{j=1}^{s}(±i)^{k_{j}}δ}{| det(1 − g)|^{\frac{1}{2}}} .

§5. Cocycles.

In this section we write the Cartan subgroup H̃^{m,r,s} of S̃p(2n, ℝ) in terms of a certain cocycle. Proposition 4.6 then has a very simple form.

Let c̃_{n}(,) be the “Rao” cocycle on Sp(2n, ℝ), i.e. the normalized ±1-valued cocycle of [11]. Henceforth we let S̃p(2n, ℝ) be the metaplectic cover of Sp(2n, ℝ) realized explicitly via the Rao cocycle, i.e. as pairs (g; ϵ) (g ∈ Sp(2n, ℝ), ϵ = ±1) with multiplication (g, ϵ)(g′, ϵ′) = (gg′, ϵϵ′c̃_{n}(g, g′)).

From the preceding results it is clear the calculation of the character of the oscillator representation restricted to a Cartan subgroup H̃^{m,r,s} comes down to a calculation on S̃L(2, ℝ) and S̃p(4, ℝ). We have written the Cartan subgroup H̃^{m,r,s} of S̃p(2n, ℝ) in terms of the exponential map ẽxp. To express our results in terms of the Rao cocycle, it is necessary to write ẽxp in these terms. It turns out to be simpler to write the cover H̃^{m,r,s} in terms of a closely related cocycle ĉ_{H}. The calculation of ẽxp reduces as well to SL(2, ℝ) and Sp(4, ℝ).

Let x : Sp(2n, ℝ) → ±1 be the map of ([11], Lemma 5.1). Recall some notation from Lemma 4.2: t_{θ} = exp(X_{θ}) is an element of the compact Cartan subgroup T of SL(2, ℝ). For x ∈ ℝ^{∗} let α_{x} = diag(x, \frac{1}{x}), an element of the split Cartan subgroup A of SL(2, ℝ). In the notation of Lemma 4.3, a_{c} = α_{e^{c}}. Then x(α_{y}) = sgn(y), and

x(t_{θ}) = \{ \begin{matrix}−sgn(sin(θ)) & sin(θ) ≠ 0, \\ cos(θ) & sin(θ) = 0. \end{matrix}  (5.1)

Recall ζ was defined in §4 (Definition 4.1).

Lemma 5.2.

For t_{θ}, t_{ϕ} ∈ T,

c̃_{1}(t_{θ}, t_{ϕ}) = ζ(\frac{θ}{2})ζ(\frac{ϕ}{2})ζ(\frac{θ+ϕ}{2}).  (5.2)(a)

and

ẽxp(X_{θ}) = (t_{θ}; ζ(\frac{θ}{2})).  (b)

For α_{x}, α_{y} ∈ A,

c̃_{1}(α_{x}, α_{y}) = (x, y)_{ℝ}.  (c)

In the notation of Lemma 4.3, let ã_{c} (c = c_{0} + πik) be an element of Ã, and let a_{c} = p(ã_{c}) = α_{e^{c}}. Then

ã_{c} = (a_{c}; (−1)^{k(k−1)/2}).  (d)

The restriction of the cocycle c̃_{2}(,) to the complex Cartan subgroup H^{1,0,0} of Sp(4, ℝ) is trivial. Let g̃ = t̃· exp(X) as in Lemma 4.4, with image g in H̃^{1,0,0}. Then

g̃ = (g; δ)  (e)

Proof. By [11] (the Remark following Corollary 5.8),

c̃_{1}(t_{θ}, t_{ϕ}) = (x(θ), x(ϕ))_{ℝ}(−x(θ)x(ϕ), x(θ + ϕ))_{ℝ}  (5.3)(a)

where we have written x(θ) = x(t_{θ}). Statement (a) follows from a straightforward but tedious calculation. It is easier, however, to proceed by an indirect method which has the advantage of proving (a) and (b) simultaneously.

Write ẽxp(X_{θ}) = (t_{θ}, ψ(θ)) for some ψ : ℝ → ±1. To prove (b) we need to show ψ(θ) = ζ(\frac{θ}{2}). The condition that ẽxp is a group homomorphism from 𝔱_{0} to T̃ gives

ψ(θ)ψ(ϕ) = c̃_{1}(t_{θ}, t_{ϕ})ψ(θ + ϕ)  (b)

for all θ, ϕ. Set θ = ϕ to conclude

ψ(θ) = c̃_{1}(t_{\frac{θ}{2}} , t_{\frac{θ}{2}} )  (c)

which by (a) and elementary properties of the Hilbert symbol [13] equals

= sgn(x(\frac{θ}{2})x(θ)).  (d)

Write e^{i \frac{θ}{2}} = x + iy. For generic θ we conclude

ψ(θ) = sgn((−y)(−2xy)) = sgn(x) = sgn cos(\frac{θ}{2}) = ζ(\frac{θ}{2})  (e)

proving (5.2)(b) for generic θ.

This argument holds provided xy ≠ 0. If y = 0 (5.3)(d) gives ψ(θ) = x(x^{2}−y^{2}) = x = cos(\frac{θ}{2}) = ζ(\frac{θ}{2}), whereas x = 0 gives ψ(θ) = (−y)(−x^{2} + y^{2}) = −y = − sin(\frac{θ}{2}) = ζ(\frac{θ}{2}). This proves (5.2)(b), and (a) follows from (5.3)(b).

Statement (5.2)(c) and the triviality of the cocycle on H^{1,0,0} follow immediately from ([11], Corollary 5.5(2)).

Statements (5.2)(d) and (e) follow from (b), since the covering is essentially on T. Thus with ã_{c} = ẽxp(πk)exp(c_{0}) as usual, (b) implies ã_{c} = (a_{c}; ζ(\frac{πk}{2} )), and then (d) follows from the definition of ζ. Finally if δ = 1, (e) is immediate since the cocycle is trivial and H^{1,0,0} is connected. For δ = −1 it follows from the fact that ẽxp(𝔥^{0,1,0}(π, π)) = (−I; −1). This completes the proof.

Remark 5.4. Note that although ζ(\frac{θ}{2}) is not well defined on T, the product of the three terms on the right hand side of (5.2)(a) is well defined. If θ → ζ(\frac{θ}{2}) were well-defined on T, (5.2)(a) would say c is a co-boundary, which of course is not the case. The cocycle on T may be lifted to the universal cover ℝ →^{exp} T, and (5.2)(a) can be interpreted as an expression for this lift as a coboundary, pushed down to T.

It is an immediate consequence of (5.2)(b,d,e) and Proposition 4.6 that on the compact or split Cartan subgroups of S̃L(2, ℝ) or on the complex Cartan subgroup H̃^{1,0,0} of S̃p(4, ℝ) we have the simple formula

[θ(ψ)_{even} − θ(ψ)_{odd}](g; ϵ) = \frac{ϵ}{\sqrt[ψ]{det(1 + g)}}.  (5.5)

With the appropriate cocycle on H^{m,r,s}, (5.5) extends to S̃p(2n, ℝ). As noted earlier, the calculations of the precedings sections reduce to SL(2, ℝ) and Sp(4, ℝ), ultimately because this is true of the second formula in Theorem 3.11. The natural cocycle on H^{m,r,s} is therefore “diagonal”.

Recall (§3) H^{m,r,s} is isomorphic to ℂ^{∗m}×S^{1r}×ℝ^{∗s}, and we introduce coordinates on H^{m,r,s} accordingly: for z_{i} ∈ ℂ^{∗}, u_{i} ∈ S^{1} and x_{i} ∈ ℝ^{∗}, we let

H^{m,r,s}(z_{1}, …, z_{m}, u_{1}, …, u_{r}, x_{1}, …, x_{s})  (5.6)(a)

be the corresponding element of H^{m,r,s}. More precisely for w_{i} ∈ ℂ, θ_{i} ∈ ℝ and c_{i} ∈ ℝ + πiZ,

H^{m,r,s}(e^{w_{1}}, …, e^{w_{m}}, e^{iθ_{1}}, …, e^{iθ_{r}}, e^{c_{1}}, …, e^{c_{s}})  (5.6)(b)

is the exponential of the element

𝔥^{m,r,s}(w_{1}, …, w_{m}, θ_{1}, …, θ_{r}, c_{1}, …, c_{s})  (5.6)(c)

of the complex Cartan subalgebra 𝔥^{m,r,s}.

Definition 5.7. Let H^{m,r,s} ≃ ℂ^{∗m} × S^{1r} × ℝ^{∗s} be one of our chosen Cartan subgroups of Sp(2n, ℝ). We define a cocycle ĉ_{H}(,) on H to be the product of the cocycles on each factor obtained by restriction from SL(2, ℝ) (to S^{1}, ℝ^{∗}) and Sp(4, ℝ) (to ℂ^{∗}). Define Ĥ^{m,r,s} to be the two-fold cover of H^{m,r,s} defined by ĉ_{H}(,).

To be precise, suppose

g = H^{m,r,s}(z_{1}, …, z_{m}, e^{iθ_{1}}, …, e^{iθ_{r}}, x_{1}, …, x_{s})

and similarly g′. Then

ĉ_{H}(g, g′) = ∏_{i=1}^{r} c̃_{1}(t_{θ_{i}}, t_{θ′_{i}}) ∏_{i=1}^{s} c̃_{1}(α_{x_{i}}, α_{x′_{i}})

with c̃_{1}(,) given explicitly in Lemma 5.2.

We will see Ĥ^{m,r,s} is isomorphic to H̃^{m,r,s} (Lemma 5.9).

Definition 5.8. Write g ∈ H^{m,r,s} as in (5.6)(a-c). For 1 ≤ i ≤ r + s let

x_{i}(g) = \{ \begin{matrix}x(t_{θ_{i}}) & j = 1, …, r, \\ x(α_{x_{j−r}}) & j = r + 1, …, r + s. \end{matrix}

Define

τ(g) = ∏_{1≤i<j≤r+s}(x_{i}(g), x_{j}(g))_{ℝ}.

Finally define a map ϕ : Ĥ^{m,r,s} → S̃p(2n, ℝ) by

ϕ(g; ϵ) = (g; ϵτ(g)).

The image of ϕ is H̃^{m,r,s}.

Lemma 5.9.

(1) For g, h ∈ H^{m,r,s},

c̃_{n}(g, h) = ĉ_{H}(g, h)τ(g)τ(h)τ(gh),

in other words c̃_{n}(,) restricted to H^{m,r,s} and ĉ_{H} differ by the coboundary of τ.

(2) ϕ is an isomorphism between Ĥ^{m,r,s} and H̃^{m,r,s}.

(3) Write g̃ = t̃ · exp(X) = ẽxp(Z)exp(X) ∈ H̃^{m,r,s} as in (3.15)(a-c), and let g = p(g̃). Then

ϕ^{−1}(g̃) = (g; ϵ)  (5.10)(a)

with

ϵ = ∏_{i=1}^{r} ζ(\frac{θ_{i}}{2} ) ∏_{i=1}^{s}(−1)^{k_{i}(k_{i}−1)/2}δ.  (5.10)(b)

Proof. We consider (1). If H^{m,r,s} has only one factor, i.e. H^{m,r,s} ≃ ℂ^{∗}, S^{1} or ℝ^{∗} this is precisely how ĉ_{H}(,) was defined. In the case of two factors (1) becomes ([11], Corollary 5.6). The general case follows by induction on the number of factors. Statement (2) is an immediate consequence of (1), and (3) follows from (5.2)(b,d,e) and the fact that the cocycle ĉ_{H} is diagonal.

Via ϕ we identify H̃^{m,r,s} and Ĥ^{m,r,s}, and write the character of the oscillator representation as a function on the regular elements of Ĥ^{m,r,s}.

Proposition 5.11. Let g̃ = (g; ϵ) be an element of Ĥ^{m,r,s}, with g a regular element of H^{m,r,s}. Then

[θ(ψ)_{even} − θ(ψ)_{odd}](g; ϵ) = \frac{ϵ}{\sqrt[ψ]{det(1 + g)}}.

Proof. This follows immediately from Proposition 4.6 and Lemma 5.9.

The following formula directly on H̃^{m,r,s} ⊂ S̃p(2n, ℝ) (without recourse to Ĥ^{m,r,s}) follows immediately from Lemma 5.9.

Corollary 5.12. Let (g; ϵ) be a regular element of H̃^{m,r,s} ⊂ S̃p(2n, ℝ). Then

[θ(ψ)_{even} − θ(ψ)_{odd}](g; ϵ) = \frac{ϵτ(g)}{\sqrt[ψ]{det(1 + g)}}.

§6. The Complex Case.

The corresponding results for the oscillator representation of the complex group G = Sp(2n, ℂ) are much easier, and easily read off from [14]. For the sake of completeness we include the statement.

Write ω = ω_{even} ⊕ ω_{odd} for the oscillator representation of G. This is unique up to isomorphism. Write θ = θ_{even} + θ_{odd} for its character.

We identify a Cartan subgroup H of G with C^{∗n} as usual. We also identify 𝔥_{0} = Lie(H) and 𝔥_{0}^{∗} with ℂ^{n}. For λ_{1}, λ_{2} ∈ ℂ^{n} satisfying λ_{1} − λ_{2} ∈ 2πiℤ^{n} the standard module X(λ_{1}, λ_{2}) [20] is defined. Let λ = (n − \frac{1}{2}, …, \frac{3}{2}, \frac{1}{2}). Then (λ, λ) is the infinitesimal character of the oscillator representation, and X(λ, λ) contains ω_{even} as a constituent. The coherent continuation action of the Weyl group W of type C_{n} is defined on representations with this infinitesimal character, and w · X(λ, λ) = X(wλ, λ).

Proposition 6.1.

∑_{w∈W} w · X(λ, λ) = ω_{even} − ω_{odd}

This may be proved in the same way as Theorem 2.3. For another proof see ([14], §II.2). As an immediate consequence we conclude:

Proposition 6.2. For g a regular element of H,

(θ_{even} − θ_{odd})(g) = \frac{∑_{w∈W ×W} sgn(w)e^{w(λ,λ)}}{∑_{w∈W ×W} sgn(w)e^{w(ρ,ρ)}} (g).

Finally proceeding as in §4 we obtain:

Proposition 6.3. For g a regular element of H,

(θ_{even} − θ_{odd})(g) = \frac{1}{|det(1 + g)|}.

Alternatively we may embed Sp(2n, ℂ) in Sp(4n, ℂ), taking H ≃ ℂ^{∗n} to H^{n,0,0}. Noting that the oscillator representation of S̃p(4n, ℝ) restricts to the the oscillator representation of Sp(2n, ℂ), this follows from Proposition 4.5. This also follows by taking −g in ([14], §II, Théorème 1).

References.

[1] J. Adams, “Lifting of Characters on Orthogonal and Metaplectic Groups,” preprint.

[2] J. Adams & D. Vogan, “L-Groups, Projective Representations, and the Langlands Classification,” Trans. Amer. Math. Soc. 113 (1992), 45–138.

[3] T. Enright, R. Howe & N. Wallach, “A Classification of Unitary Highest Weight Modules,” Proceedings of the Univ. of Utah Conference, Boston (1983).

[4] Harish-Chandra, “The characters of semisimple Lie groups,” Trans. Amer. Math. Soc. 83 (1956), 98–163.

[5] H. Hecht, “The characters of some representations of Harish-Chandra,” Math. Ann. 219 (1976), 213–226.

[6] H. Hecht & W. Schmid, “Characters, Asymptotics, and 𝔫-Homology of Harish-Chandra Modules,” Acta Math. 151 (1983), 49–151.

[7] R. Howe, “The Character of the Oscillator Representation,” preprint.

[8] J. Humphreys, Introduction to Lie Algebras and Representation Theory, Springer-Verlag, New York, 1972.

[9] R. Langlands & D. Shelstad, “On the Definition of Transfer Factors,” Math. Ann. 278 (1987), 219–271.

[10] S. Martens, “The characters of the holomorphic discrete series,” Proc. Nat. Acad. Sci. U.S.A. 72 (1975), 3275–3276.

[11] R. Ranga Rao, “On some explicit formulas in the theory of the Weil representation,” Pacific J. Math. 157 (1993), 335–371.

[12] W. Schmid, “On the Characters of the Discrete Series,” Invent. Math. 30 (1975), 47–144.

[13] J .P. Serre, A course in arithmetic, springer-Verlag, New York, 1973.

[14] P. Torasso, “Sur le caractère de la représentation de Shale-Weil de Mp(n, ℝ) et Sp(n, ℂ),” Math. Ann. 252 (1980), 53–86.

[15] D. Vogan, Representations of Real Reductive Lie Groups, Birkhäuser, Boston, 1981.

[16] D. Vogan, “Singular Unitary Representations,” in Non-commutative harmonic analysis and Lie groups, J. Carmona & M. Vergne, eds., Lecture notes in mathematics #880, 1981, 506–535.

[17] D. Vogan, Unitary Representations of Reductive Lie Groups, Princeton University Press, Princeton, 1987.

[18] N. Wallach, Real Reductive Groups, Academic Press, Boston, 1992.

[19] N. Wallach, “The Analytic Continutation of the Discrete Series I,” Trans. Amer. Math. Soc. 251 (July, 1979), 1–17.

[20] D. P. Zhelobenko, Analyse harmonique sur les groupes semi-simples complexes, Nauka, Moscow, 1974.

Math Dept, University of Maryland, College Park, MD 20742