THE PLANCHEREL FORMULA FOR THE

HOROCYCLE SPACE AND GENERALIZATIONS

Ronald L. Lipsman

Department of Mathematics University of Maryland College Park, Maryland 20742 USA

Abstract. The Plancherel formula for the horocycle space, and several generalizations, is derived within the framework of quasi-regular representations which have monomial spectrum. The proof uses only machinery from the Penney-Fujiwara distribution-theoretic technique; no special semisimple harmonic analysis is needed. The Plancherel formulas obtained include the spectral distributions and the intertwining operators that effect the direct integral decomposition of the quasi-regular representation.

1. Introduction. In [6] and [7] I laid out a program which indicated that the Plancherel formula for a homogeneous space G/H should be within reach whenever the spectrum of the induced representation τ = Ind_{H}^{G}1 is generically monomial. The latter means that almost all of the irreducible unitary representations π that occur in τ are representations induced from a closed subgroup B of G by a character. For any such monomial representation π, I prescribed a natural distribution β_{π,B} on the space of π and computed its matrix coefficient. The Plancherel formula is then a direct integral decomposition of the canonical cyclic distribution α_{τ} associated to τ (see below) into the spectral distributions β_{π,B}. The definition of β_{π,B} and the computation of its matrix coefficient in [6] were rigorous. However, the derivation of the Plancherel formula in [7] was heuristic. It was left as a challenge to render the heuristic proof rigorous for different categories of homogeneous spaces G/H having monomial spectrum.

This challenge was taken up in several papers. In [5], a special algebraic symmetric space was treated. In [6], completely solvable G and abelian symmetric spaces G/H were considered. In [8], the situation was generalized to polynomial spectrum — meaning the generic spectrum of τ is induced from finite-dimensional representations — and Strichartz symmetric spaces were handled. Other cases are considered in [1], [7] and [9], but in every instance so far G has been a non-semisimple group. In this paper we shall derive the Plancherel formula for several monomial and polynomial homogeneous spaces of a semisimple Lie group G. The most familiar example is the horocycle space G/MN, where G = KAN is an Iwasawa decomposition, and M is the centralizer of A in K. The Plancherel formula for that space has been known for a long time [2]. But our proof will be much easier — coming about as an almost immediate consequence of the results of [6]. We shall also employ the same method to examine two kinds of generalizations. First of all, we shall consider homogeneous spaces G/H where H is one of the canonical groups between N and the minimal parabolic subgroup MAN, other than MN. Second, we shall expand the horocycle example to include non-minimal parabolics.

Supported by NSF # DMS-92-25063 .

The main results of the paper are explicit Plancherel formulas, including multiplicity, spectral distributions, and the intertwining operator, for the homogeneous spaces G/MN where P = MAN is any parabolic group; and also for the spaces G/P, G/AN and G/N when P = MAN is a minimal parabolic subgroup. These results are proven in: Theorems 2.2, 2.5 and Corollaries 2.3, 2.4; Theorem 2.6; Theorem 3.2; and Theorem 3.3, respectively. The cases G/MN and G/P are known. G/AN and G/N apparently are not to be found in the literature — although almost any semisimple specialist could have derived them. The point is that the known (and presumed) proofs of these Plancherel formulas have been obtained in the past by techniques special to semisimple groups. But in fact, the results of this paper show that all of them — since they manifest monomial, or at worst polynomial, spectrum — can be derived from the general theory of such homogeneous spaces developed in [6] [7] and [8].

2. Monomial Spectrum. We recall the results from [6] (to which we refer the reader for basic terminology and notation). Let G be a Lie group and H ⊂ G a closed subgroup. We fix right Haar measures dg, dh on G and H. We write Δ_{G}, Δ_{H} for the modular functions of G, H respectively (i.e. the derivative of right Haar measure with respect to left). We set Δ_{H,G} = Δ_{H}/Δ_{G}, a positive character on H. We select a smooth function q = q_{H,G} on G satisfying q(e) = 1, q(hg) = Δ_{H,G}(h)q(g). If χ is a unitary character of H, the induced representation π_{χ} = Ind_{H}^{G}χ acts on the space (2.1)

C_{c}^{∞}(G, H, χ) = {f ∈ C^{∞}(G) : f(hg) = χ(h)f(g), |f| compactly supported mod H}

by the formula

(2.2)  π_{χ}(g)f(x) = f(xg)[q(xg)/q(x)]^{1/2}.

The action (2.2) extends naturally from (2.1) to a unitary representation. Indeed, there is a quasi-invariant measure dġ on H\G defined as follows. Any f ∈ C_{c}(G, H) can be written

f(g) = ∫_{H} F (hg) dh, F ∈ C_{c}(G);

then

∫_{H\G} f(g) dġ := ∫_{G} F (g)q(g) dg.

The formula (2.2) defines the unitary action of G on

L^{2}(G, H, χ) := the closure of C_{c}^{∞}(G, H, χ) in the norm [∫_{H\G} |f|^{2} dġ]^{1/2}.

When χ = 1, we write τ = π_{1} and refer to τ as the quasi-regular representation. (Note that q and dġ determine each other canonically.) If H needs to be specified, we write τ = τ_{H}.

Now suppose τ is type I. Then it has a direct integral decomposition into irreducible unitary representations

τ = ∫ ^{⊕} π dμ(π).

The basic assumption we make at this point is that μ − a.a. π are monomial, that is induced from a character. Focus on one such π momentarily: π = π_{χ} = Ind_{B}^{G}χ, B a closed subgroup of G, χ a character of B. Choose db and q_{B,G} and realize π_{χ} in L^{2}(G, B, χ). Suppose further that:

(I) BH is closed (II) q_{H∩B,H}q_{H∩B,B} ≡ 1 on H ∩ B (III) χ ≡ 1 on H ∩ B. With these assumptions, we can define the Penney distributions whose matrix coeffi-

cients form the critical components of the Plancherel formula. For any unitary representation π of G, we set H_{π}^{∞} = the C^{∞} vectors in the space H_{π} on which π acts, and H_{π}^{−∞} = the conjugate-dual space of distributions. If π = π_{χ}, we know

C_{c}^{∞}(G, B, χ) ⊂ L^{2}(G, B, χ)^{∞} ⊂ C^{∞}(G, B, χ).

I now summarize some of the main results of [6] — namely Theorem 2.1 and Prop. 2.2 therein. Since all of our applications in this paper will be to semisimple G, I shall state the results under the assumption that G is unimodular. In that case, I abbreviate q_{H} = q_{H,G}, q_{B} = q_{B,G}.

Theorem 2.1.

(i) The distribution α_{τ} : f → f̄(e) on L^{2}(G, H)^{∞} := L^{2}(G, H, 1)^{∞} is cyclic and relatively invariant under the action of H with modulus q_{H}^{−1/2}.

(ii) For ω ∈ D(G) := C_{c}^{∞}(G), the distribution τ(ω)α_{τ} is a smooth function ω_{H}, called the smooth form of α_{τ} and given by the formula τ(ω)α_{τ} (g) =

ω_{H}(g) := q_{H}^{−1/2}(g) ∫_{H} ω(g^{−1}h^{−1})q_{H}^{−1/2}(h) dh, ω ∈ D(G).

(iii) The matrix coefficient of α_{τ} is ⟨τ(ω)α_{τ} , α_{τ} ⟩ = ω_{H}(e) = ∫_{H} ω(h^{−1})q_{H}^{−1/2}(h) dh. (iv) If π_{χ} = Ind_{B}^{G}χ, the distribution

(2.3)  β_{χ} : f → ∫_{H∩B\H} f̄q_{B}^{1/2}q_{H}^{−1/2}q_{H∩B,H}^{−1} dḣ, f ∈ C_{c}^{∞}(G, B, χ)

is well-defined and relatively invariant under the action of H with modulus q_{H}^{−1/2}.

(v) The smooth function (or smooth form) π_{χ}(ω)β_{χ} is given by

π_{χ}(ω)β_{χ}(g) = ∫_{H∩B\B} ω_{H}(bg)χ̄(b)q_{B}^{−1/2}(bg)q_{H}^{1/2}(bg)q_{H∩B,B}^{−1}(b) dḃ, ω ∈ D(G).

(vi) The matrix coefficient ⟨π_{χ}(ω)β_{χ}, β_{χ}⟩ of β_{χ} is given by

∫_{H∩B\H} ∫_{H∩B\B} ω_{H}(bh)χ̄(b)q_{B}^{−1/2}(b)q_{H}^{1/2}(h^{−1}bh)q_{H∩B,B}^{−1}(b)q_{H∩B,H}^{−1}(h) dḃ dḣ,

where ω ∈ D^{+}(G) := the positive linear combinations of functions of the form ω_{1}^{∗} ∗ ω_{1}, ω_{1} ∈ D(G).

The reader is urged to consult [6]. But at this point let us recall exactly what the Penney-Fujiwara Plancherel formula (PFPF for short) is. It is a decomposition of the matrix coefficient of α_{τ} into matrix coefficients of distributions β attached to irreducible unitary representations. Namely, it asserts that one can find a measurable family X of irreducible unitary representations, a positive regular Borel measure μ on X, and distributions β_{π} attached to π ∈ X so that

(2.4)  ⟨τ(ω)α_{τ} , α_{τ} ⟩ = ∫_{X}⟨π(ω)β_{π}, β_{π}⟩ dμ(π), ω ∈ D(G).

An immediate consequence (see [6,Prop.3.2]) is that the map

τ(ω)α_{τ} → {π(ω)β_{π}}

is the intertwining operator for the direct integral decomposition τ = ∫_{X}^{⊕} π dμ(π) — at least if the multiplcity is finite. In this paper, we shall find the Plancherel measure and establish the formula (2.4) for various semisimple homogeneous spaces. Remarks. There are several subtleties that can arise in this venture that have been addressed in previous work.

(1) The integrals in the definition (2.3) of the Penney distributions β_{χ} are welldefined because of Conditions (II) and (III). The integrals converge because of Condition (I). One expects that the integrals actually converge for all vectors f ∈ L^{2}(G, B, χ)^{∞}. The proof of that fact may involve a very delicate argument depending heavily on the structure of the homogeneous space (see [5] or [1] e.g.).

(2) The second subtle point is that even if one cannot demonstrate convergence of the distribution integrals (2.3) for all C^{∞} vectors, Proposition 3.3 of [6] shows that if one can prove (2.4), then μ − a.a. distributions have unique extensions from C_{c}^{∞}(G, B, χ) to the full space of C^{∞} vectors. The value of β on a non-compactly supported test function could conceivably be given by something other than the absolutely convergent integral, but no such example is known.

(3) A further complication is caused by the fact that the assumption (I) may sometimes be violated (see e.g. [1]). In that case, one cannot assert that for f ∈ C_{c}^{∞}(G, B, χ) the integrand in the definition (2.3) of β_{χ} is automatically in C_{c}(H, H ∩ B). Thus the convergence of the integral is problematic even for test functions. Experience has shown ([1]) that, even without condition (I), the integral in (2.3) is still convergent — not only for f ∈ C_{c}^{∞}(G, B, χ) — but ultimately for all C^{∞} vectors. On the other hand, the integral specifying the smooth form — and so also the intertwining operator — may prove to be more troublesome. This scenario is illustrated in section 2b below.

(4) It is worth reviewing what is “chosen” and what is “canonical” in the setup. The Haar measures are chosen and fixed. The q functions q_{B}, q_{H} are chosen, but any alteration affects the L^{2} space since q and dġ are paired. Finally, there is freedom to choose q_{H∩B,H}, but the measure q_{H∩B,H}^{−1}(h)dḣ is canonical — ditto with q_{H∩B,B}^{−1}(b)dḃ. In this way we see that everything in Theorem 2.1 is — if not canonical, then at least — completely natural.

Until further notice G is a connected semisimple Lie group with finite center. Also we fix a maximal compact subgroup K and an Iwasawa decomposition G = KAN. Let M be the centralizer of A in K. Then P = MAN is a minimal parabolic subgroup. If we fix Haar measures dm, da, dn on the unimodular groups M, A, N, then dmdadn is left Haar measure on P, and ∫_{N} f(n) dn = e^{2ρ(log a)} ∫_{N} f(ana^{−1}) dn, where ρ is one-half the sum of the positive roots (on 𝔞 with respect to 𝔫). In particular, e^{2ρ(log a)}dmdadn is right Haar measure on P and Δ_{P} (man) = e^{2ρ(log a)} is the modular function.

2a. Generalized Horocycle Spaces. Now we select H = MN, a unimodular group. The homogeneous space G/H = G/MN is the horocycle space [2]. We derive, as a consequence of Theorem 2.1, its Plancherel theorem.

Theorem 2.2. For τ = τ_{MN} = Ind_{MN}^{G}1, we have the Plancherel formula

(2.5)  ⟨τ_{MN}(ω)α_{τ} , α_{τ} ⟩ = ∫Â⟨π_{λ}(ω)β_{λ}, β_{λ}⟩ dλ, ω ∈ D(G),

where π_{λ} is the irreducible principal series representation π_{λ} = Ind_{MAN}^{G}1 × λ × 1, λ ∈ Â, and dλ is the Lebesgue measure on Â dual to da.

Proof. We begin with the representation-theoretic computation of a direct integral decomposition of τ. In it, we use induction in stages, and the fact that direct integrals commute with induction:

τ = Ind_{MN}^{G}1 = Ind_{MAN}^{G}Ind_{MN}^{MAN}1

= Ind_{MAN}^{G} ∫ ⊕ ; Â 1 × λ × 1 dλ

= ∫ ⊕ ; Â Ind_{MAN}^{G}1 × λ × 1 dλ

= ∫ ⊕ ; Â π_{λ} dλ. (2.6)

Now we apply Theorem 2.1 with H = MN and B = B_{λ} = MAN. Then H ∩ B = H. Therefore we have: q_{H} ≡ 1 on G; q_{H∩B,H} ≡ 1 on H; q_{B} = Δ_{B} = e^{2ρ(log a)} on B; and q_{H∩B,B} = 1 on MN. We define q_{B} on G via the Iwasawa decomposition q_{B}(ank) = \frac{Δ_{B}}{Δ_{G}} (ank) = e^{2ρ(log a)}; and we choose q_{H,B} on MAN according to q_{H,B}(man) = \frac{Δ_{MN}}{Δ_{MAN}} (man) = e^{−2ρ(log a)}. At this point we observe that conditions (I)–(III) are satisfied.

Next we must identify the quasi-invariant measure on H\B = MN\MAN associated to the above choice of q_{H,B}. In fact it is relatively invariant since q_{H,B} = Δ_{B}^{−1} is a character on B. We can be even more precise. Since the right Haar measure on B = MAN is db = Δ_{B}dmdadn, we have

∫_{B} f(b)q_{H,B}(b) db = ∫_{MAN} f(man)q_{H,B}(man)Δ_{B}(man) dmdadn
= ∫_{MAN} f(man) dmdadn
= ∫_{A} (∫_{MN} f(mna)e^{−2ρ(log a)} dmdn) da.

Therefore if we identify the homogeneous space MN\MAN to A we see that dḃ = e^{−2ρ(log a)}da is the relatively invariant measure on H ∩ B\B = H\B. Consequently we have

q_{H,B}^{−1}(b)dḃ = da.

(This reinforces our previous remark that q_{H∩B,B}^{−1}(b)dḃ is canonical.)

Now we continue with the application of the general formulas of Theorem 2.1 to this specific situation. First of all, the Penney distributions collapse to

β_{λ} : f → ∫_{H∩B\H} f̄q_{B}^{1/2}q_{H}^{−1/2}q_{H∩B,H}^{−1} dḣ = f̄(e).

Next, we evaluate their smooth form and matrix coefficients to be:

π_{λ}(ω)β_{λ}(g) = ∫_{H∩B\B} ω_{H}(bg)λ̄(b)q_{B}^{−1/2}(bg)q_{H}^{1/2}(bg)q_{H∩B,B}^{−1}(b) dḃ
= ∫_{A} ω_{H}(ag)λ̄(a)q_{B}^{−1/2}(ag) da;

and ⟨π_{λ}(ω)β_{λ}, β_{λ}⟩

= ∫_{H∩B\H} ∫_{H∩B\B} ω_{H}(bh)λ̄(b)q_{B}^{−1/2}(b)q_{H}^{1/2}(h^{−1}bh)q_{H∩B,B}^{−1}(b)q_{H∩B,H}^{−1}(h) dḃdḣ

= ∫_{A} ω_{H}(a)λ̄(a)e^{−ρ(log a)} da.

The proof of Theorem 2.2 follows as a consequence of the Plancherel formula on A. Indeed,

∫Â⟨π_{λ}(ω)β_{λ}, β_{λ}⟩ dλ = ∫Â ∫_{A} ω_{H}(a)λ̄(a)e^{−ρ(log a)} dadλ

= ω_{H}^{ρ}(e) where ω_{H}^{ρ} is the test function ω_{H}^{ρ}(a) = ω_{H}(a)e^{−ρ(log a)}

= ω_{H}(e),

which establishes formula (2.5).

As explained in [6], the map ω → Ω = ω_{H} maps D(G) onto D(G, H), and so Theorem 2.2 supplies the intertwining operator for the direct integral decomposition.

Corollary 2.3. The map

Ω(g) → {Ω_{λ}(g)}_{λ∈Â}
(2.7)  Ω_{λ}(g) = ∫_{A} Ω(ag)λ̄(a)q_{B}^{−1/2}(ag) da, Ω ∈ D(G, MN)

extends uniquely to a unitary operator L^{2}(G, MN) → ∫ ⊕ ; Â L^{2}(G, B, λ) dλ which effects the direct integral decomposition (2.6).

Remarks.

(5) For the record we recall how to compute q_{B}^{−1/2}(ag). One writes the Iwasawa decomposition G = ANK in coordinates g = α(g)ν(g)𝔨(g), and then q_{B}^{−1/2}(ag) = e^{−ρ(log α(ag))}.

(6) For emphasis, we remark that Theorem 2.2 and Corollary 2.3 follow completely from the general monomial PFPF found in [6] and specific semisimple structure theory. No additional semisimple harmonic analysis is necessary.

Next we observe that, although all the constituent representations π_{λ} in the direct integral decomposition (2.6) are irreducible, they are not pairwise inequivalent. We rewrite formula (2.5) to take that into account, and to count the multiplicity.

Let Â′ = {λ ∈ Â : wλ ≠ λ, ∀w ∈ W := Norm_{K}(A)/M}. W is the Weyl group which acts naturally on M and A. It acts simply transitively on the set Â′, which is a dense open co-null subset of Â. Two representations π_{λ_{1}}, π_{λ_{2}} are equivalent iff λ_{1} and λ_{2} are in the same W -orbit. Therefore we have

Corollary 2.4.

⟨τ_{MN}(ω)α_{τ} , α_{τ} ⟩ = ∫_{Â′/W} #(W)⟨π_{λ}(ω)β_{λ}, β_{λ}⟩ dλ, ω ∈ D(G).

In particular, the quasi-regular representation τ has uniform multiplicity equal to the order of the Weyl group.

One big advantage of this method for deriving the Plancherel formula for the horocycle space is that it does not really depend on the compactness of M. That is, it works for any parabolic subgroup, not just a minimal parabolic subgroup. Let P = MAN be any parabolic and set H = MN, τ = Ind_{H}^{G}1. A review of the preceding proofs reveals that nowhere was the minimality of P (or the compactness of M) utilized. The exact same arguments apply to yield

Theorem 2.5. If we write π_{λ} = Ind_{MAN}^{G}1 × λ × 1 and β_{λ} : f → f̄(e), then we have

⟨τ(ω)α_{τ} , α_{τ} ⟩ = ∫Â⟨π_{λ}(ω)β_{λ}, β_{λ}⟩ dλ, ω ∈ D(G).

Moreover, the map

Ω(g) → {Ω_{λ}}_{λ∈Â}

(2.8)  Ω_{λ}(g) = ∫_{A} Ω(ag)λ̄(a)q_{B}^{−1/2}(ag) da, Ω ∈ D(G, MN),

extends uniquely to a unitary operator L^{2}(G, MN) → ∫ ⊕ ; Â L^{2}(G, B, λ) which effects the direct integral decomposition of τ. The multiplicity is uniformly #(W), where W = Norm(A)/Cent(A).

Remarks.

(7) The notation in Theorem 2.5 presumes that B = B_{λ} = P = MAN and g = μ(g)α(g)ν(g)𝔨(g), where in the decomposition G = P K, the A component is uniquely determined. Furthermore, q_{B}(g) = e^{2ρ(log α(g))}.

(8) In both the minimal and non-minimal cases, we clearly have that β_{λ} is defined for all C^{∞} vectors — because the integral collapses. Furthermore, the integral (over A) expressing π_{λ}(ω)β_{λ} is convergent for ω compactly supported (since AMN = B is closed in G), and extends to the intertwining operator. But the question of exactly what the set of functions Ω for which the integral (2.8) actually converges is an interesting question in Fourier analysis.

2b. Generalized Flag Manifolds. The goal of the paper is the Plancherel formula for G/H where H is a canonical subgroup of a parabolic group P containing the nilradical N. In the previous subsection we considered H = MN. The only really “canonical” group above MN is P itself. But, for P minimal, it is a well-known fact [4] that the induced representation τ = Ind_{P}^{G}1 is actually irreducible — it has no decomposition. So what would constitute a Plancherel formula? The answer is as follows. Fix a minimal parabolic group P_{1} = MAN_{1} (the reason behind the choice of notation will be clear momentarily). Write τ_{1} = τ_{P_{1}} = Ind_{P_{1}}^{G}1, an irreducible representation. The point is that it is known that for any other parabolic P_{2} = MAN_{2}, associate (and therefore conjugate) to P_{1}, the corresponding induced representation τ_{2} = τ_{P_{2}} = Ind_{P_{2}}^{G}1 is unitarily equivalent to τ_{1}. A Plancherel formula would specify the intertwining operator between the two representations. Such a formula is well-known [3,ch. VII.4] as a consequence of classical work (of Kunze-Stein) on intertwining operators. Next, we shall see that those operators emerge from our framework naturally as well.

We take H = P_{1}, B = P_{2}. Then H ∩B = MA(N_{1} ∩N_{2}). The corresponding q functions are as follows:

q_{H} = q_{H,G} = Δ_{MAN_{1}} = e^{2ρ_{1}(log a)}
q_{B} = q_{B,G} = Δ_{MAN_{2}} = e^{2ρ_{2}(log a)}
q_{H∩B,H} = \frac{Δ_{MA(N_{1}∩N_{2})}}{Δ_{MAN_{1}}} = e^{2(ρ_{0}−ρ_{1})(log a)}
q_{H∩B,B} = \frac{Δ_{MA(N_{1}∩N_{2})}}{Δ_{MAN_{2}}} = e^{2(ρ_{0}−ρ_{2})(log a)}.

Here we write N_{0} = N_{1} ∩ N_{2} and ρ_{j}, j = 0, 1, 2 for one-half the sum of the corresponding positive roots. Now the fact that P_{1} and P_{2} are conjugate guarantees that 2ρ_{0} = ρ_{1} + ρ_{2}. Therefore condition (II) is satisfied. But it is unlikely that condition (I) will hold if the two parabolics are not equal. This is the situation referred to in Remark 3 in the last section. Nevertheless, we can cope with this problem as follows.

The relevant H-invariant Penney distribution (on the space of τ_{2}) is

β_{2} : f → ∫_{H∩B\H} f̄q_{B}^{1/2}q_{H}^{−1/2}q_{H∩B,H}^{−1} dḣ

= ∫_{N_{0}\N_{1}} f̄dṅ_{1}, (2.9)

because all the q functions live only on A. Moreover, the integral (2.9) clearly converges for any C^{∞} vector f. Indeed, if we write V for the nilradical of the opposed parabolic to P_{2}, and we realize τ_{2} = Ind_{P_{2}}^{G}1 in L^{2}(V), then any C^{∞} vector will be a Schwartz function on V. Therefore it will be Schwartz on N_{0}\N_{1} and convergence is assured. Finally, we remark that condition (III) is trivially satisfied.

Next we evaluate the smooth form of β_{2}:

τ_{2}(ω)β_{2}(g) = ∫_{H∩B\B} ω_{H}(bg)q_{B}^{−1/2}(bg)q_{H}^{1/2}(bg)q_{H∩B,B}^{−1}(b) dḃ
= ∫_{N_{0}\N_{2}} ω_{P_{1}}(n_{2}g)q_{B}^{−1/2}(n_{2}g)q_{H}^{1/2}(n_{2}g) dṅ_{2}.

Then we observe that the irreducibility of τ_{1} and τ_{2}, together with [10,Thm. 3.3.], implies that (up to scalar) the matrix coefficients of α_{1} = α_{τ_{1}} and β_{2} must coincide. Therefore, for an appropriate choice of Haar measures, we have

⟨τ_{1}(ω)α_{1}, α_{1}⟩ = ⟨τ_{2}(ω)β_{2}, β_{2}⟩.

This is the PFPF and thus, by the discussion after Theorem 2.1, the intertwining operator between τ_{1} and τ_{2} is formally

Ω_{1}(g) → Ω_{2}(g) = ∫_{N_{0}\N_{2}} Ω_{1}(n_{2}g)q_{B}^{−1/2}(n_{2}g)q_{H}^{1/2}(n_{2}g) dṅ_{2}
= ∫_{N_{0}\N_{2}} Ω_{1}(n_{2}g)q_{B}^{−1/2}(g)q_{H}^{1/2}(n_{2}g) dṅ_{2}.

The left N_{2}-invariance of q_{B} is clear from its definition. The conclusion is (see [3, ch. VII.4])

Theorem 2.6. The intertwining operator giving the unitary equivalence between τ_{1} and τ_{2} is specified by

(2.10)  Ω(g) → e^{−ρ_{2}(log α_{2}(g))} ∫_{N_{0}\N_{2}} Ω(n_{2}g)e^{ρ_{1}(log α_{1}(n_{2}g))} dṅ_{2}

Remark. (9) Formula (2.10) differs slightly from [3] because of different setups — we have built the equivariance condition into the group action (see (2.2)), whereas Knapp builds it into the function space. Also, although the distribution integral (2.9) converges, we encounter the usual convergence difficulties with the intertwining integral (2.10).

3. Polynomial Spectrum. We take now as our basic frame of reference the generalization of [6] found in [8]. The new assumption is that generically the spectrum of τ consists only of polynomial representations, that is, those induced from finite-dimensional representations. We retain the same notation as in section 2, except we write σ instead of χ, that is σ is a finite-dimensional representation acting on a space H_{σ}. We still have the inclusions C_{c}^{∞}(G, B, σ) ⊂ L^{2}(G, B, σ)^{∞} ⊂ C^{∞}(G, B, σ) for the spaces on which π_{σ} = Ind_{σ}^{G}σ acts; but the functions in these spaces are H_{σ}-valued instead of scalar-valued. The statements of conditions (I) and (II) are unchanged, but (III) is stated in the form

(III) σ|_{H∩B} contains a fixed vector.

As in the transcription of the key results in [6] to Theorem 2.1, we capture all the basic results of [8] in a single statement.

Theorem 3.1. Let ξ ∈ H_{σ} be a σ(H ∩ B)-invariant vector.

(i) The distribution

(3.1)  β_{ξ} : f → ∫_{H∩B\H}⟨ξ, f(·)⟩q_{B}^{1/2}q_{H}^{−1/2}q_{H∩B,H}^{−1} dḣ, f ∈ C_{c}^{∞}(G, B, σ)

is well-defined and relatively invariant under the action of H with modulus q_{H}^{−1/2}.

(ii) The smooth vector-valued function (or smooth form) π_{σ}(ω)β_{ξ} is given by

π_{σ}(ω)β_{ξ}(g) = ∫_{H∩B\B} ω_{H}(bg)σ(b)^{−1}ξ [q_{B}^{−1/2}(bg)q_{H}^{1/2}(bg)q_{H∩B,B}^{−1}(b)] dḃ, ω ∈ D(G).

(iii) The matrix coefficient ⟨π_{σ}(ω)β_{ξ}, β_{ξ}⟩ of β_{ξ} is given by

∫_{H∩B\H} ∫_{H∩B\B} ω_{H}(bh)⟨ξ, σ(b)ξ⟩q_{B}^{−1/2}(b)q_{H}^{1/2}(h^{−1}bh)q_{H∩B,B}^{−1}(b)q_{H∩B,H}^{−1}(h) dḃ dḣ,

for ω ∈ D^{+}(G).

Next we shall consider canonical subgroups between N and P without any M component. But to preserve the polynomial spectrum hypothesis, we shall require that M be compact. So in the next two subsections P = MAN will be a minimal parabolic subgroup.

3c. Rossi-Vergne Spaces. In this subsection we shall consider H = AN. We start with the representation-theoretic decomposition of τ = τ_{AN} = Ind_{AN}^{G}1. In fact

τ = Ind_{AN}^{G}1 = Ind_{MAN}^{G}Ind_{AN}^{MAN}1

= Ind_{MAN}^{G} ∑_{σ∈M̂}^{⊕}(dim σ)(σ × 1 × 1) (by the Peter-Weyl Theorem on M)

= ∑_{σ∈M̂}^{⊕}(dim σ)Ind_{MAN}^{G}σ × 1 × 1.

For the typical σ, the representation π^{σ} = Ind_{MAN}^{G}σ × 1 × 1 will be irreducible; but not for all σ. A perfect example is supplied by G = SL(2, ℝ). M contains two elements, and for σ = 1, π^{σ} is irreducible, but for σ ≠ 1, it is not. We shall ignore the reducibility of the decomposable representations π^{σ}. We give our Plancherel formula in terms of the principal series representations π^{σ}. Further decomposition into irreducibles is determined by the Rgroups [3,ch. XIV.9], and we leave that (fairly sophisticated) portion of the semisimple theory to the interested reader.

There is still the matter of equivalences — which indeed may occur. In fact, π^{σ} ≅ π^{σ′} ⇔ ∃w ∈ W ∋ w · σ ≅ σ′. Let us assume that a cross-section for M̂/W in M̂ has been selected. We abuse notation by purposefully confusing M̂/W with the cross-section. Denote W_{σ} = {w ∈ W : w · σ ≅ σ}, σ ∈ M̂/W. Then

(3.2)  τ_{AN} = ∑_{M̂/W}^{⊕}(dim σ)#(W_{σ})π^{σ}

gives us a direct sum decomposition into inequivalent representations of G with multiplicity counted. Now, bearing in mind that some of the π^{σ} may break up — but that such a break up is always finite and multiplicity-free (the R-groups are finite abelian) — formula (3.2) does specify the multiplicity, if not the precise spectrum.

Let us describe the q functions in this scenario. We have: H = AN, B = MAN, H∩B = H. Therefore H∩B\B = M (with its normalized Haar measure). Clearly q_{H}(g) = q_{B}(g) = e^{2ρ(log a)}, if g = ank. Furthermore, q_{H∩B,H} = 1, and q_{H∩B,B} = q_{AN,MAN} = 1 also. In particular, conditions (I)–(III) hold here.

Now for any fixed σ, let ξ_{1}^{σ}, …, ξ_{dim σ}^{σ} be an orthonormal basis in H_{σ}. Then from Theorem 3.1, the Penney distributions, smooth form and matrix coefficients are:

β_{ξ_{j}^{σ}} : f → ⟨ξ_{j}^{σ}, f(e)⟩
π^{σ}(ω)β_{ξ_{j}^{σ}}(g) = ∫_{M} ω_{AN}(mg)σ(m)^{−1}ξ_{j}^{σ} dm
⟨π^{σ}(ω)β_{ξ_{j}^{σ}}, β_{ξ_{j}^{σ}}⟩ = ∫_{M} ω_{AN}(m)⟨ξ_{j}^{σ}, σ(m)ξ_{j}^{σ}⟩ dm.

Then, by the (matrix coefficient version of the) Peter-Weyl Theorem on M, we have

∑_{σ∈M̂} ∑_{j=1}^{dim σ}⟨π^{σ}(ω)β_{ξ_{j}^{σ}}, β_{ξ_{j}^{σ}}⟩ = ∑_{σ∈M̂} ∑_{j=1}^{dim σ} ∫_{M} ω_{AN}(m)⟨ξ_{j}^{σ}, σ(m)ξ_{j}^{σ}⟩ dm
= ω_{AN}(e).

Hence we have proven

Theorem 3.2.

⟨τ_{AN}(ω)α_{τ} , α_{τ} ⟩ = ∑_{σ∈M̂} ∑_{j=1}^{dim σ}⟨π^{σ}(ω)β_{ξ_{j}^{σ}}, β_{ξ_{j}^{σ}}⟩

= ∑_{σ∈M̂/W} #(W_{σ}) ∑_{j=1}^{dim σ}⟨π^{σ}(ω)β_{ξ_{j}^{σ}}, β_{ξ_{j}^{σ}}⟩, ω ∈ D(G).

Moreover, the intertwining operator is Ω(g) → {Ω_{σ}^{j}(g)}, where

Ω_{σ}^{j}(g) = ∫_{M} Ω(mg)σ(m)^{−1}ξ_{j}^{σ} dm, Ω ∈ D(G, AN).

3d. Whittaker Spaces. The final canonical subgroup we investigate is H = N. The abstract Plancherel theory for the Whittaker space G/N is contained in the following computation.

τ_{N} = Ind_{N}^{G}1 = Ind_{MAN}^{G}Ind_{N}^{MAN}1

= Ind_{MAN}^{G} ∑_{σ∈M̂}^{⊕} dim σ ∫ ⊕ ; Â σ × λ × 1 dλ

= ∑_{σ∈M̂}^{⊕} dim σ ∫ ⊕ ; Â π_{λ}^{σ} dλ,

where π_{λ}^{σ} = Ind_{MAN}^{G}σ × λ × 1 is a general principal series representation. Now we are back in the situation of generic irreducibility. If we restrict λ to lie in Â′, it does not matter if σ is fixed by any elements in the Weyl group — the representation π_{λ}^{σ} will still be irreducible. Therefore, we have

τ_{N} = Ind_{N}^{G}1 = ∫_{Â′/W}^{⊕} ∑_{σ∈M̂}(dim σ)#(W)π_{λ}^{σ} dλ.

The next order of business is the q functions. We have H = N, B = MAN, H ∩B = H. Then clearly

q_{H,G} = q_{H} = 1 and q_{H∩B,H} = 1, on G and H, respectively.

We choose the other two q functions as in section 2a, namely

q_{B,G} = e^{2ρ(log a)}, for g = ank and
q_{H,B}(man) = e^{−2ρ(log a)}.

One verifies readily that q_{H,B}^{−1}(b)dḃ = dmda. Conditions (I)–(III) are satisfied. We select the vectors ξ_{j}^{σ} as in section 3c. Then the Penney distributions become

β_{λ,ξ_{j}^{σ}} : f → ⟨ξ_{j}^{σ}, f(e)⟩.

The smooth form and the matrix coefficients are computed to be:

π_{λ}^{σ}β_{λ,ξ_{j}^{σ}}(g) = ∫_{MA} ω_{N}(mag)λ̄(a)σ(m)^{−1}ξ_{j}^{σ}[q_{B}^{−1/2}(mag)] dmda;

⟨π_{λ}^{σ}β_{λ,ξ_{j}^{σ}}, β_{λ,ξ_{j}^{σ}}⟩ = ∫_{MA} ω_{N}(ma)λ̄(a)⟨ξ_{j}^{σ}, σ(m)ξ_{j}^{σ}⟩e^{−ρ(log a)} dmda.

If we sum over j and σ, integrate over Â′/W, and use the technique that occurs at the end of the proof of Theorem 2.2, we obtain

Theorem 3.3.

⟨τ_{N}(ω)α_{τ} , α_{τ} ⟩ = ∫_{Â′/W} #(W) ∑_{σ∈M̂} ∑_{j=1}^{dim σ}⟨π_{λ}^{σ}(ω)β_{λ,ξ_{j}^{σ}}, β_{λ,ξ_{j}^{σ}}⟩ dλ.

We leave it to the reader to write down the intertwining operator.

References

1. B. Currey Smooth decomposition of finite multiplicity monomial representations for a class of completely solvable homogeneous spaces, (to appear).

2. S. Helgason, A duality for symmetric spaces with applications to group representations, Advances in Math 5 (1970), 1-154.

3. A. Knapp, Representation Theory of Semisimple Lie Groups, Princeton University Press, Princeton, 4. B. Kostant, On the existence and irreducibility of certain series of representations, Lie Groups and

their Representations (Summer School of the Bolyai János Math. Soc.), Halsted Press, New York, 1975, pp. 231-329.

5. R. Lipsman, The Penney-Fujiwara Plancherel formula for symmetric spaces, Progress in Math 82 (1990), 135-145, The Orbit Method in Representation Theory, Birkhäuser.

6. R. Lipsman, The Penney-Fujiwara Plancherel formula for abelian symmetric spaces and completely solvable homogeneous spaces, Pacific J. Math. 151 (1991), 265-295.

7. R. Lipsman, The Penney-Fujiwara Plancherel formula for homogeneous spaces, Representation Theory of Lie Groups and Lie Algebras, World Scientific Press, 1992, pp. 120-139.

8. R. Lipsman, The Plancherel formula for homogeneous spaces with polynomial spectrum, Pacific J. Math. 159 (1993), 351-377.

9. R. Lipsman The Penny-Fujiwara Plancherel formula for Gelfand pairs, (to appear).

10. R. Lipsman Orbital symmetric spaces and finite multiplicity, (to appear).