A UNIFIED APPROACH TO CONCRETE

PLANCHEREL THEORY OF HOMOGENEOUS SPACES

Ronald L. Lipsman

Department of Mathematics University of Maryland College Park, Maryland 20742 USA

Abstract. A new paradigm for concrete Plancherel analysis on homogeneous spaces is established wherein the distinction between finite and infinite multiplicity is de-emphasized. A unified treatment—incorporating the Penney-Fujiwara formulation for finite multiplicity and the Bonnet formulation for infinite multiplicity—is presented. The heretofore unrecognized similarities in the two theories are uncovered and emphasized. The unified theory is illustrated with explicit computations of the Plancherel formula for a certain class of homogeneous spaces of semidirect product groups.

1. Introduction

Let G be a Lie group, H ⊂ G a closed subgroup, and τ = τ_{G,H} = Ind_{H}^{G}1 the quasi-regular representation. Suppose τ is type I. Then there is a unique direct integral decomposition

(1.1)  τ = ∫ ⊕ ; Ĝ n_{π}π dμ(π).

That assertion is what is usually referred to as the “abstract Plancherel Theorem” for the homogeneous space G/H. By concrete Plancherel theory, we mean a more specific analytic formula that provides detailed information not only on the multiplicity function n_{π} and the Plancherel measure μ, but also on an intertwining operator that effects the direct integral decomposition. The pursuit of such a formula has been carried out in two mutually exclusive cases according as n_{π} is finite or infinite μ-a.e. The former pursuit has been under the rubric of the Penney-Fujiwara Plancherel formula (PFPF)—see [5], [12], [13], [20]; the latter under that of the Bonnet Plancherel formula (BPF)—see [3]. The two cases have been viewed as rather disjoint in spirit; with much progress having been made in the former [1], [6], [14], [15], [18], [19], and considerably less in the latter [8], [10]. Our approach here is that they should be viewed in a unified way, that the PFPF is just a special case of the BPF in the lucky happenstance of finite multiplicity, and that the distinction between the two cases is not really a matter of Plancherel theory but of orbital constructs or of other invariants associated to the homogeneous space G/H.

Supported by NSF # DMS-92-25063 .

Suppose now that G is the connected component of a (real) algebraic group, and H is an algebraic subgroup. Then compelling evidence is presented in [17] for the likelihood that there are only three possibilities for the multiplicity function: (1.2i) n_{π} is finite and bounded (μ-a.e.) (1.2ii) n_{π} is finite, but unbounded (μ-a.e.) (1.2iii) n_{π} is infinite (μ-a.e.). In [17] it is conjectured that (1.2i) is equivalent to either: (1.3i) the algebra D(G/H) of invariant differential operators on G/H is abelian; or (1.3ii) (The Orbit Condition) generically on 𝔥^{⊥} = {φ ∈ 𝔤^{∗} : φ(𝔥) = 0}, the H-orbits on

the intersection of a G-orbit with 𝔥^{⊥} are open. In case (1.2i), one utilizes the PFPF to express the Plancherel theory in a distributiontheoretic setting. In (1.2iii), the BPF establishes a setting involving nuclear operators. That they are two sides of the same coin is already evident from the nilpotent case [8]. In the nilpotent arena, case (1.2ii) does not occur. In general, the second case does occur and, although the PFPF is the governing mechanism, no useful orbital constructs will be available if the conjecture in [17] is valid. Thus, the existence or non-existence of orbital constructs in the Plancherel theory of G/H may be related to the nature of the multiplicity function, but it does not really measure whether the PFPF or BPF scenario should be paramount. This suggests that we shouldn’t emphasize the difference between the two Plancherel theories, but rather concentrate on their similarities. In this paper we do exactly that: philosophically in this section; theoretically in §2, where we present some general results; and concretely in later sections, where we specialize to a specific type of homogeneous space (in which, incidentally, situation (1.2ii) also does not occur).

The category we consider is Strichartz homogeneous spaces—meaning G = V ⋊ R, a semidirect product of a real algebraic group R by a normal vector subgroup V, H = U ⋊ S of the same type with S ⊂ R, U ⊂ V —and in this work, we assume: for generic λ ∈ U^{⊥} = {λ ∈ V̂ : λ(U) = 1}, the stability group R_{λ} is trivial. This has the effect of forcing the generic representations in the spectrum of τ to be monomial. Hence the results of [14], [15] can be brought to bear. (Note that the case of primary interest is that of R reductive, but we don’t need to make that assumption to obtain our results.) In §3 we present general information on such spaces. In §§4 and 5 we examine the finite and infinite multiplicity cases, respectively. Finally in §6, we give some indication of other categories of homogeneous spaces for which we believe this approach to a unified Plancherel theory could profitably be pursued.

The main results of the paper are Theorems 4.1 and 5.1—the PFPF and BPF for the Strichartz homogeneous spaces introduced above (in the finite and infinite multiplicity cases respectively)—and the collection of important Remarks 2.3 that unifies the two types of Plancherel theories.

2. The Unified Approach

We begin with statements of the PFPF and BPF. The context is always that τ = τ_{G,H} is a type I representation, resulting in a well-determined Plancherel decomposition (1.1). To insure that [3] is valid, we shall also assume in this paper that G is unimodular.

We begin by fixing a choice of right Haar measures dg, dh on G and H respectively. We write Δ_{H} for the modular function of H (i.e. the derivative of right Haar measure with respect to left). By assumption, Δ_{G} ≡ 1. We choose once and for all a smooth function q = q_{H,G} satisfying q(e) = 1, q(hg) = Δ_{H}(h)q(g), ∀h ∈ H, g ∈ G. Now suppose π is a unitary representation of G acting on a Hilbert space H_{π}. We write H_{π}^{∞} to denote the Frechet space of C^{∞} vectors and H_{π}^{−∞} for its anti-dual of distributions (i.e. conjugate linear functionals). As usual we set ⟨v, α⟩ = \overline{α(v)}, v ∈ H_{π}^{∞}, α ∈ H_{π}^{−∞}, a sesquilinear form. π(G) acts on both H_{π}^{∞} and H_{π}^{−∞}, and it’s a well-known fact that π(D(G))H_{π}^{∞} ⊂ H_{π}^{−∞}, if D(G) is the test space of compactly supported, infinitely differentiable functions on G. Given the direct integral decomposition (1.1), it is known from [9] and [20] that

H_{τ}^{∞} = ∫ ^{⊕} n_{π}H_{π}^{∞} dμ(π) H_{τ}^{−∞} = ∫ ^{⊕} n_{π}H_{π}^{−∞} dμ(π).

We refine the latter of these as follows. Set (H_{π}^{−∞})^{H,q^{−1/2}} = {α ∈ H_{π}^{−∞} : π(h)α = q^{−1/2}(h)α, ∀h ∈ H\}. It is established in [14] that the distribution α_{τ} : f → \overline{f(e)} lies in

(H_{τ}^{−∞})^{H,q^{−1/2}}, and its matrix coefficient ⟨τ(ω)α_{τ} , α_{τ} ⟩, ω ∈ D(G), is calculated there as follows:

τ(ω)α_{τ} (g) = ω_{H}(g), where
(2.0)  ω_{H}(g) = q^{−1/2}(g) ∫_{H} ω(g^{−1}h^{−1})q^{−1/2}(h) dh, ω ∈ D(G);

and by a routine change of variable (see [14,Prop. 2.2])

⟨τ(ω)α_{τ} , α_{τ} ⟩ = ω_{H}(e) = ∫_{H} ω(h^{−1})q^{−1/2}(h) dh = ∫_{H} ω(h)q^{−1/2}(h) dh.

Now we can state

Theorem 2.1 (PFPF). [5] [20] Suppose the multiplicity in (1.1) is finite μ-a.e. Then for μ-a.a. π, there exist n_{π} linearly independent distributions β_{π}^{1}, …, β_{π}^{n_{π}} ∈ (H_{π}^{−∞})^{H,q^{−1/2}} such that

(2.1)  ⟨τ(ω)α_{τ} , α_{τ} ⟩ = ∫Ĝ ∑_{j=1}^{n_{π}}⟨π(ω)β_{π}^{j}, β_{π}^{j}⟩ dμ(π), ω ∈ D(G).

As observed in [14,Prop. 3.2], it follows that

ω_{H}(g) ↪ {π(ω)β_{π}^{j}}_{π∈Ĝ,j=1,...,n_{π}}

is an intertwining operator effecting the direct integral decomposition (1.1). As explained earlier, by a concrete Plancherel formula for a homogeneous space (or category of homogeneous spaces) G/H, we mean explicit expressions for: μ, the distributions β_{π}^{j}, and the smooth vectors π(ω)β_{π}^{j} (see e.g. [12], [13], [14]).

Next, let’s state

Theorem 2.2 (BPF). [3] For μ-a.a. π, there exist positive nuclear operators U_{π} : H_{π}^{∞} → H_{π}^{−∞} such that

(2.2)  ω_{H}(e) = ∫Ĝ Tr[π(ω)U_{π}] dμ(π), ω ∈ D(G).

Furthermore, the pair (μ, {U_{π}}_{π∈Ĝ}) is unique up to positive scalars c(π); i.e. any other pair (μ′, \{U′_{π}\}_{π∈Ĝ}) satisfying formula (2.2) must be related to the original by \frac{dμ}{dμ′} = c(π), U_{π} = U′_{π}/c(π), for some positive Borel function c(π).

We now make a series of key remarks and observations intended to bind together the two theorems.

Critical Remarks 2.3.

(1) As indicated earlier, we assume G is unimodular; but there is no need to require unimodularity for H in [3]. Right Haar measure on H is still a positive distribution on G and Bonnet’s theory applies.

(2) The nuclear operators U_{π} actually take values in the appropriate subspace of Hinvariant distributions. In fact, if we replace U_{π} by π(h)U_{π} in (2.2) and use the easily verified facts that

π(ω)π(h) = π(ω^{h^{−1}}), ω^{h}(g) = ω(gh) and (ω^{h})_{H} = q^{−1/2}(h)ω_{H},

then the uniqueness part of Theorem 2.2 insures that

U_{π} (H_{π}^{∞}) ⊆ (H_{π}^{−∞})^{H,q^{−1/2}}.

Equality is problematic—see (4) below.

(3) Therefore, Theorem 2.1 is really a special case of Theorem 2.2. Namely, if the rank of the operator U_{π} is finite μ-a.e., then—since the β_{π}^{j} must come from the range of U_{π}—if we set

(2.3)  V_{π}v = ∑_{j=1}^{n_{π}}⟨v, β_{π}^{j}⟩β_{π}^{j},

we clearly obtain

Tr[π(ω)V_{π}] = ∑_{j}⟨π(ω)β_{π}^{j}, β_{π}^{j}⟩.

By uniqueness, we may take U_{π} = V_{π}. In summary, the multiplicity n_{π} is the rank of U_{π}; and in that case, the Penney distributions β_{π}^{j} form a basis of distributions in the image of U_{π}, and U_{π} is defined as in (2.3). Momentarily, we shall see how to go the other way, i.e. recover the β_{π}^{j} from U_{π}.

(4) The assertion that for μ-a.a. π, the image of U_{π} exhausts (H_{π}^{−∞})^{H,q^{−1/2}} is precisely the assertion of (distributional) Frobenius reciprocity—namely, that

n_{π} = dim(H_{π}^{−∞})^{H,q^{−1/2}}.

Of course, by Penney’s fundamental estimate [20], we know

n_{π} ≤ dim(H_{π}^{−∞})^{H,q^{−1/2}}.

Whether this equality can ever fail on a set of positive μ-measure is still an open question. What I am observing here is that—in the finite multiplicity situation—this question is exactly the same as the question of whether the operators

U_{π} : H_{π}^{∞} → (H_{π}^{−∞})^{H,q^{−1/2}}

are surjective.

(5) The operator U_{π} being positive means that ⟨v, U_{π}v⟩ ≥ 0, ∀v ∈ H_{π}^{∞}. This is completely evident in the case of finite multiplicity from (2.3). In infinite multiplicity, the requisite positivity is sometimes helpful in seeing how to construct the nuclear operators (as e.g. in Remark (3) above).

(6) Now suppose we are in the finite multiplicity situation and we “know” the Bonnet operators U_{π}. Then we can ask: how do we recover the Penney distributions β_{π}^{j} from the nuclear operators U_{π}? First, we observe that ker U_{π} = ∩_{v∈H_{π}^{∞}} ker U_{π}v. In fact, if w is a vector in the intersection, then ⟨w, U_{π}v⟩ = 0, ∀v ⇒ conj⟨v, U_{π}w⟩ = 0, ∀v ⇒ w ∈ ker U_{π}. Conversely, if U_{π}w = 0, then ⟨v, U_{π}w⟩ = conj⟨w, U_{π}v⟩ = 0, ∀v ⇒ w is in the intersection. Hence U_{π} defines a positive-definite sesquilinear form on the finite-dimensional space H_{π}^{∞}/W_{π}, W_{π} = ker U_{π}. Clearly, if we choose a collection of vectors v_{j} that diagonalize the form, and set β_{π}^{j} = U_{π}v_{j}, then that will serve as the desired collection. It is unique up to a unitary change of variables in the finite-dimensional vector space.

(7) Now the mapping ω_{H} ↪ {π(ω)U_{π}}_{π∈Ĝ} is indeed an intertwining operator effecting the direct integral decomposition (1.1)—provided it’s interpreted properly. First, we observe that the map is well-defined. If ω_{H} = 0, then it is an easy exercise to see that if we replace ω by ω^{∗} ⋆ ω, the vanishing condition ω_{H} = 0 is preserved. But the resulting positivity and the Plancherel formula (2.2) (or (2.1)) show then that π(ω)U_{π} must be zero μ-a.e. Next, select a basis {β_{π}^{j}}_{1≤j≤n_{π}} associated to U_{π} as in (6). Of course, in the finite multiplicity case, there is no ambiguity (aside from at worst a unitary transformation). Then the prescription

ω_{H} ↪ {{π(ω)β_{π}^{j}}_{1≤j≤n_{π}}}_{π∈Ĝ}

is an intertwining operator (see [12], [13]). In the case of infinite multiplicity, we choose a maximal (countable) set of eigenvectors for the sesquilinear form ⟨v, U_{π}w⟩ and let β_{π}^{j} be their images under U_{π}. Then

ω_{H} ↪ {{π(ω)β_{π}^{j}}_{1≤j≤∞}}_{π∈Ĝ}

is an intertwining operator in the sense that it extends to an isometry

L^{2}(H\G) ↪ ∫ ⊕ ; Ĝ H_{π} ⊗ K_{π} dμ(π), K_{π} = closure(range U_{π}),

that intertwines the actions of the quasi-regular representation and ∫^{⊕} π ⊗ 1 dμ(π). It’s reasonable to denote ω_{H} ↪ {π(ω)U_{π}}_{π∈Ĝ} as the intertwining operator—in the sense that π(ω)U_{π} is a Hilbert-Schmidt operator from H_{π} to K_{π} that functions exactly as in the classical Plancherel Theorem [4,Thm. 18.8.1] (where H is trivial and U_{π} = Id).

3. Strichartz homogeneous spaces with generically trivial stabilizers

Henceforth, we assume that our Lie group G is of the form G = V ⋊ R, where V is a normal vector subgroup. Although the primary case of interest is when R is reductive, we shall not need to make that assumption in what follows. However, we will assume G is unimodular. Note that we write the normal subgroup on the left so that R is acting on it from the right. In fact, all of our group actions will be written on the right so that we can use the environment established in [11]. Let H = U ⋊ S be a closed subgroup of G of the same form, with U ⊂ V, S ⊂ R. We shall assume all groups are algebraic, although in truth, it is really enough to assume that G is type I and U^{⊥}/S is countably separated, U^{⊥} = {λ ∈ V̂ : λ(U) = 1}. We further assume that generically on U^{⊥}, the stability group R_{λ} is trivial. We fix choices of right Haar measure dr, ds, du, dv.

Now the irreducible unitary representations of semidirect products G = V ⋊R are given in general by

π_{λ,σ} = Ind_{V R_{λ}}^{G}λσ, λ ∈ V̂ , σ ∈ R̂_{λ}.

Because of our basic assumption on the stabilizers, the generic irreducible representations that lie over U^{⊥} are monomial

π_{λ} = Ind_{V}^{G}λ.

Moreover, π_{λ} ≅ π_{λ′} ⇔ λ, λ′ are in the same R-orbit. Then one knows [15]—in the abstract—how to decompose the quasi-regular representation τ = τ_{G,H} = Ind_{H}^{G}1:

τ = ∫_{U^{⊥}/S}^{⊕} π_{λ} dλ̇ = ∫_{(U^{⊥}·R)/R}^{⊕} n_{λ}π_{λ} dλ̈,

where

n_{λ} = #(λ · R ∩ U^{⊥})/S,

and dλ̇, dλ̈ are pseudo-images of the Lebesgue measure dλ on U^{⊥}—which is uniquely specified by the prior choices of du and dv. Now in the algebraic situation in which we have placed ourselves, it is a fact that the generic S-orbits on the varieties λ · R ∩ U^{⊥} are either open or not. Moreover, in the latter case, there is an upper bound on the possible number of orbits [16]. Thus we see that the numbers n_{λ} are either finite and bounded or uniformly infinite.

Next we make some specific choices of modular functions. We adopt the format of [11]. Let A ⊂ B be Lie groups with right Haar measures da, db. If B is normal in A, then there is a uniquely determined modulus δ_{A,B} satisfying

(3.1)  ∫_{B} f(b) db = δ_{A,B}(a) ∫_{B} f(aba^{−1}) db.

More generally, if A acts as a group of automorphisms of B, formula (3.1) continues to hold (with aba^{−1} replaced by b · a^{−1}). If A = B ⋊ C, then the modular function of A is given by

Δ_{A}(bc) = Δ_{B}(b)Δ_{C}(c)q_{C,B}(c).

Now if B is any closed subgroup of A (not necessarily normal), then we can choose a smooth function q = q_{A,B} that satisfies q(e) = 1 and q(ba) = \frac{Δ_{B}(b)}{Δ_{A}(a)}q(a), b ∈ B, a ∈ A. Then there is uniquely determined a quasi-invariant measure dȧ on B\A that satisfies

∫_{A} f(a)q(a) da = ∫_{B\A} ∫_{B} f(ba) dbdȧ

(see [11,§3]). Any other function q′ satisfying the same two conditions as q must be related to it by q′ = qρ, where ρ is a smooth function obeying ρ(ba) = ρ(a), b ∈ B, a ∈ A. In that case, the associated quasi-invariant measure is ρ^{−1} times the original.

Now refer to the setup in [14], or [15]. The group V will play the role of a polarization group and V ∩ H = U. Therefore, we need to choose the q functions for the homogeneous spaces: H\G, V \G, U\H, and U\V. In fact, we set:

q_{U,V} ≡ 1  q_{V,G} ≡ 1  q_{U,US} ≡ 1.

Since the groups U, V, G are unimodular, the first two choices are completely canonical. Comments are in order on the last, and on the choice of q_{H,G}. A homogeneous space of the form U\US actually has a relatively invariant measure because the positive character Δ_{U}/Δ_{US} = Δ_{U}/Δ_{U}Δ_{S}δ_{S,U} = 1/Δ_{S}δ_{S,U} is trivial on U. It extends from U to a positive character on all of US, and there are two natural ways to do that: set q_{U,US} ≡ 1 or set q_{U,US}(us) = Δ(s)^{−1}δ_{S,U}(s)^{−1}. If we do the former, then the relatively invariant measure dṡ on U\US ∼ S is right Haar measure ds. This is because duds is right Haar measure on US and

∫_{US} f(us) duds = ∫_{S} (∫_{U} f(us) du) ds.

But if we make the latter choice, in fact if we choose any smooth function ρ(s) and take q(us) = ρ(s), then

∫_{US} f(us)ρ(s) duds = ∫_{S} (∫_{U} f(us) du) ρ(s) ds = ∫_{S} ∫_{U} f(us) dudṡ.

Therefore, q(s)^{−1}dṡ = ds regardless of the choice of ρ.

Finally, we comment on the choice of q_{H,G}. We have Δ_{G} = 1, so Δ_{R}δ_{R,V} ≡ 1. We fix a choice of q_{S,R}. Next we observe that on S we have δ_{S,U}/δ_{R,V} = δ_{S,U\V}^{−1} = δ_{S,U^{⊥}} by duality. But since Δ_{H}/Δ_{G} = (Δ_{S}/Δ_{R})δ_{S,U^{⊥}} (again on S), we see that having fixed a choice of q_{S,R}, a choice of q_{H,G} is the same as a choice of an extension of δ_{S,U^{⊥}} to R. Regardless of these choices we have the following important

Lemma 3.1. On the group S, we have the equalities (i) Δ_{S} = δ_{S,U^{⊥}}

(ii) q_{H,G} = q_{S,R}δ_{S,U^{⊥}} = Δ_{S}^{2}/Δ_{R}.

Proof. Item (i) is a consequence of [15,formula (6.1)] and the triviality of S_{λ}. Statement (ii) follows immediately from the preceding paragraph and (i).

This lemma will prove critical in the next two sections. But before we get to that, let’s make a further observation on the generic representations of G. The representations π_{λ} are realized in the space L^{2}(G, H, λ) consisting of the L^{2}-completion of the space of functions f : G → ℂ, f(vg) = λ(v)f(g), v ∈ V, g ∈ G, that are compactly supported mod V, and have finite norm [∫_{V \G} ∥f(g)∥^{2} dġ]^{1/2}. The action of the group is right translation. But of course V \G ∼ R and the invariant measure dġ is precisely dr. Therefore we may realize the representations π_{λ} on L^{2}(R, dr), in which realization the group action becomes

π_{λ}(r_{1})f(r) = f(rr_{1}), r, r_{1} ∈ R;
π_{λ}(v)f(r) = λ(rvr^{−1})f(r), r ∈ R, v ∈ V.

We shall use this realization in the sequel.

4. Finite Multiplicity: The Penney-Fujiwara Plancherel Formula

We assume in this section that the multiplicity function n_{λ} is finite. We employ the structure and results of [14], [15]. Note that conditions (a)–(c) of [15,p. 355] are valid in this setting. Therefore, according to Theorem 2.1 in either [14] or [15], the Penney distributions that appear in the PFPF are given in general by the prescription

β : f → ∫_{H∩B\H} f̄q_{B,G}^{1/2}q_{H,G}^{−1/2}q_{H∩B,H}^{−1} dḣ.

In this instance, B = V, and with the choices prescribed in the previous section we have

(4.1)  β_{λ} : f → ∫_{S} f̄(s)q_{H,G}^{−1/2}(s) ds.

For a test function ω(g), the smooth function generated by π(ω)β is (again from [15])

π(ω)β(g) = ∫_{H∩B\B} ω_{H}(bg)\overline{λ(b)}q_{B,G}^{−1/2}(bg)q_{H,G}^{1/2}(bg)q_{H∩B,B}^{−1}(b) dḃ;

which under the present circumstances becomes

π_{λ}(ω)β_{λ}(g) = ∫_{U\V} ω_{H}(vg)\overline{λ(v)}q_{H,G}^{1/2}(vg) dv̇,

with ω_{H} defined in (2.0). Finally, the matrix coefficients ⟨π(ω)β, β⟩ are specified in [15] as

⟨π(ω)β, β⟩

= ∫_{H∩B\H} ∫_{H∩B\B} ω_{H}(bh)q_{B,G}^{−1/2}(b)q_{H,G}^{1/2}(h^{−1}bh)q_{H∩B,B}^{−1}(b)q_{H∩B,H}^{−1}(h) dḃdḣ.

In our case this devolves to

⟨π_{λ}(ω)β_{λ}, β_{λ}⟩ = ∫_{S} ∫_{U\V} ω_{H}(vs)\overline{λ}(v)q_{H,G}^{1/2}(s^{−1}vs) dv̇ds
= ∫_{S} ∫_{U\V} ω_{H}(sv)λ̄(svs^{−1})q_{H,G}^{1/2}(v)δ_{S,U\V} (s) dv̇ds
= ∫_{S} ∫_{U\V} ω_{H}(v)\overline{(λ · s)(v)}δ_{S,U^{⊥}}^{−1}(s) dv̇ds.

Therefore, we have

∫_{U^{⊥}/S}⟨π_{λ}(ω)β_{λ}, β_{λ}⟩ dλ̇ = ∫_{U^{⊥}/S} ∫_{S} ∫_{U\V} ω_{H}(v)\overline{(λ · s)(v)}δ_{S,U^{⊥}}^{−1}(s) dv̇dsdλ̇,

where the choice of the pseudo-image dλ̇ is uniquely specified as follows. The Haar measure dλ is relatively invariant under the action of S with modulus δ_{S,U^{⊥}}. Moreover the stability groups S_{λ} are generically trivial. Thus, having fixed a Haar measure ds, the relatively invariant measure on S_{λ}\S = S with modulus δ_{S,U^{⊥}} is exactly δ_{S,U^{⊥}}(s)ds. Hence there is precisely one pseudo-image dλ̇ that satisfies

∫_{U^{⊥}} g(λ) dλ = ∫_{U^{⊥}/S} ∫_{S} g(λ · s)δ_{S,U^{⊥}}^{−1}(s) dsdλ̇.

Combining the previous observations and applying Fourier inversion on U\V, we obtain

∫_{U^{⊥}/S}⟨π_{λ}(ω)β_{λ}, β_{λ}⟩ dλ̇ = ∫_{U^{⊥}} ∫_{U\V} ω_{H}(v)λ̄(v) dv̇dλ
= ω_{H}(e).

This is close, but not the final Plancherel formula—because we have not accounted for duplication. Namely, we have to take into account that points in U^{⊥} from distinct Sorbits may lie in the same R-orbit. So select a Borel cross-section C for the action of R in U^{⊥}/S. Then for λ ∈ C, the set λ · R ∩ U^{⊥} has exactly n_{λ} distinct S-orbits. Suppose λ_{0} = λ · r_{0} lies in a different S-orbit from λ. We need to compute how the distribution β_{λ_{0}} transfers over to the space of λ. Well it is easy to see that the unitary transformation T_{0} : f(r) → f(r_{0}r)Δ_{R}(r_{0})^{1/2} on L^{2}(R, dr) converts the action of π_{λ} into that of π_{λ_{0}}. Therefore, to obtain the distribution on L^{2}(R, dr) that is semiinvariant for the action of π_{λ_{0}} rather than π_{λ}, we only need form

β_{λ,r_{0}} : f → β_{λ}(T_{0}f) = ∫_{S} \overline{T_{0}f(s)}q_{H,G}^{−1/2}(s) ds
= Δ_{R}(r_{0})^{1/2} ∫_{S} f̄(r_{0}s)q_{H,G}^{−1/2}(s) ds.

So if λ_{j} = λ · r_{j}, j = 1, …, n_{λ}, form a complete set of representatives for the S-orbits in λ · R ∩ U^{⊥}, then the n_{λ} Penney distributions we need for the Plancherel formula are

(4.2)  β_{λ}^{j} : f → Δ_{R}(r_{j})^{1/2} ∫_{S} f̄(r_{j}s)q_{H,G}^{−1/2}(s) ds;

and the PFPF is

ω_{H}(e) = ∫_{(U^{⊥}·R)/R} ∑_{j=1}^{n_{λ}}⟨π_{λ}(ω)β_{λ}^{j}, β_{λ}^{j}⟩ dλ̈,

where dλ̈ is uniquely determined by

∫_{U^{⊥}/S} g(λ) dλ̇ = ∫_{(U^{⊥}·R)/R} ∑_{j=1}^{n_{λ}} g(λ · r_{j}) dλ̈.

We summarize our results in the following

Theorem 4.1. Assume that the Strichartz homogeneous space H\G has generically trivial stabilizers and finite multiplicity. Then, if we define the Penney distributions β_{λ}^{j} as in (4.2), we have the Plancherel formula

(4.3)  ω_{H}(e) = ∫_{(U^{⊥}·R)/R} ∑_{j=1}^{n_{λ}}⟨π_{λ}(ω)β_{λ}^{j}, β_{λ}^{j}⟩ dλ̈, ω ∈ D(G).

Remarks 4.2. We make two remarks parallel to those of [15, Remarks 6.3].

(1) The distribution integrals in the definition (4.1) (or in (4.2)) of β_{λ} are certainly convergent for f ∈ D(R). Moreover, we have D(R) ⊂ L^{2}(R, dr)^{∞}. I know of no instance where the distribution integrals do not converge on the whole space of C^{∞} vectors—but it may be very difficult to prove that, and the proof will likely depend on the nature of the group S.

(2) Whether the distribution integrals converge on all of L^{2}(R, dr)^{∞} or not, there is always a unique extension of β_{λ}^{j} from D(R) to L^{2}(R, dr)^{∞} so that formula (4.3) is valid (see [14,Remark 4.2]). As indicated in (1), one expects the extension to be given by the same integral formula (4.2), but there is no a priori reason why it must be so.

5. Infinite Multiplicity: The Bonnet Plancherel Formula

We assume in this section that the multiplicity function n_{λ} is identically infinite. That is, for every λ ∈ U^{⊥}, every S-orbit on λ·R∩U^{⊥} has lower dimension than that of λ·R∩U^{⊥}. Hence

τ = ∫_{U^{⊥}/S}^{⊕} π_{λ} dλ̇ = ∞ ∫_{(U^{⊥}·R)/R}^{⊕} π_{λ} dλ̈.

(Here dλ̇ is the canonical pseudo-image of dλ (as in section 4), and dλ̈ is any measure in the class of the push-forward of dλ̇ to (U^{⊥} · R)/R.)

We employ the structure and results of [3]. So we need to identify the nuclear operators

U_{λ} : H_{λ}^{∞} → (H_{λ}^{−∞})^{H,q^{−1/2}} ,

for each λ ∈ V̂ and H_{λ} as in section three. We start by fixing some measures. We set R^{λ} = {r ∈ R : λ · r ∈ U^{⊥}}, a closed subset (although likely not a subgroup) of R. Making use of the fact that S_{λ} is generically trivial on U^{⊥}, we have the disintegration

∫_{U^{⊥}} f(λ) dλ = ∫_{U^{⊥}/S} ∫_{S} f(λ · s) dṡdλ̇.

The measures dλ and dṡ are canonical and fixed (dλ is uniquely specified by the choices of Haar measure on V and U, and from section three, dṡ is precisely δ_{S,U^{⊥}}^{−1}(s)ds. Therefore, the pseudo-image dλ̇ is also canonically determined. Now disintegrate it under the natural projection

U^{⊥}/S → (U^{⊥} · R)/R.

The fibers of the latter are precisely (λ · R ∩ U^{⊥})/S = R^{λ}/S. Hence there is a pair of measures (dṙ, dλ̈) so that if we write F (λ̇) = ∫_{S} f(λ · s) dṡ, then we have the disintegration

∫_{U^{⊥}} f(λ) dλ = ∫_{U^{⊥}/S} ∫_{S} f(λ · s) dṡdλ̇
= ∫_{U^{⊥}/S} F (λ̇) dλ̇
= ∫_{(U^{⊥}·R)/R} ∫_{R^{λ}/S} F (λ̇ · r) dṙdλ̈

= ∫_{(U^{⊥}·R)/R} ∫_{R^{λ}/S} ∫_{S} f(λ · rs) dṡdṙdλ̈. (5.1)

The pair (dṙ, dλ̈) is uniquely specified by dλ̇, in the usual sense that if one member of the pair is altered by a scalar function of λ̇ then the other must be replaced by its reciprocal. So we fix a choice of a pair. Note that there is a slight abuse of notation since the fibers R^{λ}/S depend on λ; and also, while dλ̈ denotes a single measure, dṙ denotes the totality of measures on the fibers R^{λ}/S.

Now the Hilbert spaces H_{λ} are all just L^{2}(R), but the action of the group depends on

λ. We define the nuclear operators U_{λ} : H_{λ}^{∞} → (H_{λ}^{−∞})^{H,q^{−1/2}} , λ ∈ V by a smooth kernel

K_{λ}(rs, rs_{1}) = (q_{H,G}^{−1/2}(s)ds × q_{H,G}^{−1/2}(s_{1})ds_{1})Δ_{R}(r)dṙ,

so that

(5.2)  ⟨f_{1}, U_{λ}f_{2}⟩ = ∫_{R^{λ}/S} Δ_{R}(r) dṙ ∫_{S} f_{1}(rs)q_{H,G}^{−1/2}(s) ds ∫_{S} f̄_{2}(rs_{1})q_{H,G}^{−1/2}(s_{1}) ds_{1}.

This is well-defined as follows. If we replace r by rs_{2} in the kernel and then make the two appropriate left translations in the S integrals, the fact that formula (5.1) is unaffected by s_{2} comes down to the truth of the equation Δ_{R}(s)Δ_{S}^{−2}(s)δ_{H,G}(s) ≡ 1, s ∈ S. This is precisely part (ii) of Lemma 3.1.

Next we make use of the classical computation of the kernel of an induced representation. This is done in many places, but for our purposes, the most convenient realization is that of [11,Thm. 3.2]. From that reference, we see that the kernel of the operator π_{λ}(ω) is

K_{λ}(ω)(r, r_{1}) = ∫_{V} ω(r^{−1}vr_{1})λ(v) dv, ω ∈ D(G).

Then, from Grélaud [10], we know how to combine the kernels:

Tr[π_{λ}(ω)U_{λ}] = ⟨K_{λ}(ω), K_{λ}⟩

= ∫_{R^{λ}/S} Δ_{R}(r) dṙ ∫_{S} q_{H,G}^{−1/2}(s) ds ∫_{S} q_{H,G}^{−1/2}(s_{1}) ds_{1} ∫_{V} ω(s^{−1}r^{−1}vrs_{1})λ(v) dv

= ∫_{R^{λ}/S} Δ_{R}(r) dṙ ∫_{S} q_{H,G}^{−1/2}(s) ds ∫_{S} q_{H,G}^{−1/2}(s_{1}) ds_{1}δ_{R,V} (rs) ∫_{V} ω(vs^{−1}s_{1})λ(rsvs^{−1}r^{−1}) dv

= ∫_{R^{λ}/S} Δ_{R}(r)δ_{R,V} (r) dṙ ∫_{S} q_{H,G}^{−1/2}(s)δ_{S,V} (s) ds ∫_{S} q_{H,G}^{−1/2}(s_{1}) ds_{1}×

× ∫_{V} ω(vs^{−1}s_{1})λ(rsvs^{−1}r^{−1}) dv

= ∫_{R^{λ}/S} dṙ ∫_{S} q_{H,G}^{−1/2}(s)δ_{S,V} (s) ds ∫_{S} q_{H,G}^{−1/2}(s_{1}) ds_{1} ∫_{V} ω(vs^{−1}s_{1})λ(rsvs^{−1}r^{−1}) dv

= ∫_{R^{λ}/S} dṙ ∫_{S} q_{H,G}^{−1/2}(s)δ_{S,V} (s) ds ∫_{S} q_{H,G}^{−1/2}(s_{1}) ds_{1} ∫_{V} ω(v^{−1}s^{−1}s_{1})λ̄(rsvs^{−1}r^{−1}) dv

= ∫_{R^{λ}/S} dṙ ∫_{S} q_{H,G}^{−1/2}(s)δ_{S,V} (s) ds ∫_{S} q_{H,G}^{−1/2}(s_{1}) ds_{1}×

× ∫_{U\V} ∫_{U} ω(v^{−1}u^{−1}s^{−1}s_{1})λ̄(rsvs^{−1}r^{−1}) dudv̇.

In the above, in addition to employing the changes of variable v → rsv(rs)^{−1} and v → v^{−1}, we used the unimodularity of G to cancel Δ_{R}δ_{R,V}, as well as the fact that the characters in R^{λ} are trivial on U. Now motivated by the computations in the previous section, we evaluate another expression:

∫_{R^{λ}/S} ∫_{S} ∫_{U\V} ω_{H}(v)λ̄(rsvs^{−1}r^{−1}) dv̇dṡdṙ

= ∫_{R^{λ}/S} ∫_{S} ∫_{U\V} q_{H,G}^{−1/2}(v) ∫_{US} ω(v^{−1}s_{1}^{−1}u^{−1})q_{H,G}^{−1/2}(us_{1})λ̄(rsvs^{−1}r^{−1}) duds_{1}dv̇dṡdṙ

= ∫_{R^{λ}/S} ∫_{S} ∫_{U\V} ∫_{US} ω(v^{−1}u^{−1}s_{1}^{−1})q_{H,G}^{−1/2}(s_{1})δ_{S,U}(s_{1})λ̄(rsvs^{−1}r^{−1}) duds_{1}dv̇dsdṙ

= ∫_{R^{λ}/S} ∫_{S} ∫_{U\V} ∫_{US} ω(v^{−1}u^{−1}s^{−1}s_{1}^{−1})q_{H,G}^{−1/2}(s_{1})q_{H,G}^{−1/2}(s)δ_{S,U}(s_{1})δ_{S,U}(s)×

× λ̄(rsvs^{−1}r^{−1}) duds_{1}dv̇dṡdṙ

= ∫_{R^{λ}/S} ∫_{S} ∫_{U\V} ∫_{US} ω(v^{−1}u^{−1}s^{−1}s_{1})q_{H,G}^{1/2}(s_{1})Δ_{S}^{−1}(s_{1})δ_{S,U}^{−1}(s_{1})×

× λ̄(rsvs^{−1}r^{−1})q_{H,G}^{−1/2}(s)δ_{S,U}(s) duds_{1}dsdv̇dṡdṙ

= ∫_{R^{λ}/S} ∫_{S} ∫_{U\V} ∫_{US} ω(v^{−1}u^{−1}s^{−1}s_{1})q_{H,G}^{1/2}(s_{1})Δ_{S}^{−1}(s_{1})δ_{S,U}^{−1}(s_{1})×

× λ̄(rsvs^{−1}r^{−1})q_{H,G}^{−1/2}(s)δ_{S,U}(s)Δ_{S}^{−1}(s) duds_{1}dsdv̇dṙ.

In the above we have made the successive changes of variables: u → s_{1}us_{1}^{−1}, s_{1} → s_{1}s, s_{1} → s_{1}^{−1}; and replaced dṡ by q_{U,US}(s)ds = Δ_{S}(s)^{−1}ds. But Δ_{S}δ_{S,U} = Δ_{H} = q_{H,G} on S. Also, applying Lemma 3.1(i), we see that δ_{S,U}(s)Δ_{S}^{−1}(s) = δ_{S,U}(s)δ_{S,U^{⊥}}^{−1}(s) = δ_{S,U}(s)δ_{S,U\V} (s) = δ_{S,V} (s). Therefore, the final lines of the preceding two computations are identical. Combining that conclusion with formula (5.1), we arrive at

∫_{(U^{⊥}·R)/R} Tr[π_{λ}(ω)U_{λ}] dλ̈ = ∫_{U^{⊥}} ∫_{U\V} ω_{H}(v)λ̄(v) dv̇dλ
= ω_{H}(e).

We summarize our results in the following

Theorem 5.1. Assume that the Strichartz homogeneous space H\G has generically trivial stabilizers and infinite multiplicity. Then, if we define the Bonnet nuclear operators U_{λ} as in (5.2), we have the Plancherel formula

(5.3)  ω_{H}(e) = ∫_{(U^{⊥}·R)/R} Tr[π_{λ}(ω)U_{λ}] dλ̈, ω ∈ D(G).

6. Final remarks and other categories

Looking back on the computations in the last two sections, I wish to emphasize the similarities—not the differences. Indeed, the modular group theory, the abelian Fourier analysis, the measure-theoretic disintegrations—these are largely the same. The key difference is that in section four, the fiber of the map (U^{⊥}/S → U^{⊥} ·R)/R is presumed finite, whereas in section five, it is not. In the former case, we can realize the concrete Plancherel formula via the distributions β_{λ}, while in the latter we must employ the nuclear operators U_{λ}. But the former is just a very special case of the latter, and—as the main emphasis of the paper has been trying to drive home—these are just two sides of the same coin.

The category considered here—i.e. Strichartz homogeneous spaces with generically trivial stabilizers—is interesting, but it also should be viewed as illustrative. There are many other categories of homogeneous space where, depending on some geometric or measuretheoretic invariant, the quasi-regular representation can take either a finite or infinite multiplicity form. In any instance where we know the concrete Plancherel theory in the finite multiplicity case, we should be able—using this paper as a guide—to also derive the concrete Plancherel theory in the infinite multiplicity case. Fujiwara made precisely that transition for nilpotent homogeneous spaces in progressing from [5] to [8]. Here is a roster of several other natural candidates for a similar treatment. I hope to return to one or more of these at a future time.

Remark 6.1. Homogeneous spaces G/H where both infinite and finite multiplicity arise naturally:

(1) G and H both simply connected exponential solvable (see [7]);

(2) G a connected semisimple Lie group, N ⊂ H ⊂ P where P is a parabolic subgroup of G with nilradical N (see [19])

(3) G = N ⋊ H, N a Heisenberg group of dimension 2r + 1, and H a subgroup of the symplectic group Sp(r, ℝ) that is either reductive or of the form in (2) (see [2])

(4) G and H both reductive (see [21]).

References

1. Y. Benoist, Multiplicité un pour les éspaces symétriques exponentiel, Mem. Math. Soc. France 15 (1984), 1–37; see also J. Funct. Analysis, 59 (1984), 211–253.

2. C. Benson, J. Jenkins, R. Lipsman & G. Ratcliff, A geometric criterion for Gelfand pairs associated with the Heisenberg group, Pacific J. Math. (1996) (to appear).

3. P. Bonnet, Transformation de Fourier des distributions de type positif sur un groupe de Lie unimodulaire, J. Functional Analysis 55 (1984), 220–246.

4. J. Dixmier, Les C*-algèbres et leurs Représentations, Gauthier-Villars, Paris, 1964.

5. H. Fujiwara, Représentations monomiales d’un groupe de Lie nilpotente, Pacific J. Math. 127 (1987), 329–352.

6. H. Fujiwara & S. Yamagami, Représentations monomiales d’un group de Lie résolubles exponentiels, Representations of Lie Groups, vol. 14, Advanced Studies in Pure Mathematics, 1988, pp. 153–190.

7. H. Fujiwara, Représentations monomiales des groupes de Lie résolubles exponentiels, Progress in Mathematics 82 (1990), 61–84, The Orbit Method in Representation Theory, Birkhäuser, Boston.

8. H. Fujiwara La formule de Plancherel pour les représentations monomiales des groupes de Lie nilpotents, (to appear).

9. R. Goodman, Complex Fourier analysis on nilpotent Lie groups, Trans. Amer. Math. Soc. 160 (1971), 373–391.

10. G. Grélaud, La formule de Plancherel pour les espaces homogènes des groupes de Heisenberg, J. für die Reine und Ang. Math. 398 (1989), 92–100; see also Une formule de Plancherel pour les espaces homogènes des groupes de Lie nilpotents, preprint.

11. A. Kleppner & R. Lipsman, The Plancherel formula for group extensions, Ann. Scient. Ecole. Norm. Sup. 5 (1972), 459–516; see also ibid, 6 (1973), 103–132.

12. R. Lipsman, The Penney-Fujiwara Plancherel formula for symmetric spaces, Progress in Mathematics 82 (1990), 135–145, The Orbit Method in Representation Theory, Birkhauser, Boston.

13. R. Lipsman, The Penney-Fujiwara Plancherel formula for homogeneous spaces, Representation Theory of Lie Groups and Lie Algebras, World Scientific Publishing Co., Singapore, 1992, pp. 120–139.

14. R. Lipsman, The Penney-Fujiwara Plancherel formula for abelian symmetric spaces and completely solvable homogeneous spaces, Pacific J. Math. 151 (1991), 265–295.

15. R. Lipsman, The Plancherel formula for homogeneous spaces with polynomial spectrum, Pacific J. Math. 159 (1993), 351–377.

16. R. Lipsman, The multiplicity function on exponential and completely solvable homogeneous spaces, Geometriae Dedicata 39 (1991), 155–161.

17. R. Lipsman, Attributes and applications of the Corwin-Greenleaf multiplicity function, Contemporary Mathematics 177 (1994), 27–46, Representation Theory and Analysis on Homogeneous Spaces, American Mathematical Society, Providence.

18. R. Lipsman, The Penny-Fujiwara Plancherel formula for Gelfand pairs, Rocky Mtn. J. (1996) (to appear).

19. R. Lipsman, The Plancherel formula for the horocycle space and generalizations, J. Australian Math. Soc. (1996) (to appear).

20. R. Penney, Abstract Plancherel theorems and a Frobenius reciprocity theorem, J. Functional Analysis 18 (1975), 177–190.

21. G. van Dijk, Group representations on spaces of functions, Russian J. Math. Physics 2 (1994), 57–68.