THE MODULAR GROUP ACTION ON REAL SL(2)-CHARACTERS OF A ONE-HOLED TORUS

WILLIAM M. GOLDMAN

Abstract. The group Γ of automorphisms of the polynomial

κ(x, y, z) = x^{2} + y^{2} + z^{2} − xyz − 2

is isomorphic to

PGL(2, ℤ) ⋉ (ℤ/2 ⊕ ℤ/2).

For t ∈ ℝ, the Γ-action on κ^{−1}(t) ∩ ℝ^{3} displays rich and varied dynamics. The Γ-action preserves a Poisson structure defining a Γ-invariant area form on each κ^{−1}(t) ∩ ℝ^{3}. For t < 2, the action of Γ is properly discontinuous on the four contractible components of κ^{−1}(t) ∩ ℝ^{3} and ergodic on the compact component (which is empty if t < −2). The contractible components correspond to Teichmüller spaces of (possibly singular) hyperbolic structures on a torus M̄. For t = 2, the level set κ^{−1}(t) ∩ ℝ^{3} consists of characters of reducible representations and comprises two ergodic components corresponding to actions of GL(2, ℤ) on (ℝ/ℤ)^{2} and ℝ^{2} respectively. For 2 < t ≤ 18, the action of Γ on κ^{−1}(t) ∩ ℝ^{3} is ergodic. Corresponding to the Fricke space of a three-holed sphere is a Γ-invariant open subset Ω ⊂ ℝ^{3} whose components are permuted freely by a subgroup of index 6 in Γ. The level set κ^{−1}(t) ∩ ℝ^{3} intersects Ω if and only if t > 18, in which case the Γ-action on the complement (κ^{−1}(t) ∩ ℝ^{3}) − Ω is ergodic.

Date: June 9, 2003. 1991 Mathematics Subject Classification. 57M05 (Low-dimensional topology),

20H10 (Fuchsian groups and their generalizations), 30F60 (Teichmüller theory).

Key words and phrases. surface, fundamental group, character variety, representation variety, mapping class group, ergodic action, proper action, hyperbolic structure with cone singularity, Fricke space, Teichmüller space.

The author gratefully acknowledges partial support from National Science Foundation grant DMS-9803518, DMS-0103889 and a Semester Research Award from the General Research Board of the University of Maryland in Fall 1998.

Contents

Introduction 2 1. The modular group and the moduli space 8 2. Structure of the character variety 10 3. Hyperbolic structures on tori (t < 2) 21 4. Reducible characters (t = 2) 31 5. Three-holed spheres and ergodicity (t > 2) 33 References 40

Introduction

Let M be a compact oriented surface of genus one with one boundary component, a one-holed torus. Its fundamental group π is free of rank two. Its mapping class group π_{0}(Homeo(M)) is isomorphic to the outer automorphism group Out(π) of π and acts on the space of equivalence classes of representations π ⟶ SL(2). We investigate the dynamics of this action on the set of real points on this moduli space.

Corresponding to the boundary of M is an element K ∈ π which is the commutator of free generators X, Y ∈ π. By a theorem of Fricke [8, 9], the moduli space of equivalence classes of SL(2, ℂ)-representations naturally identifies with affine 3-space ℂ^{3}, via the quotient map

Hom(π, SL(2, ℂ)) ⟶ ℂ^{3}
ρ ⟼ [\begin{matrix}x \\ y \\ z \end{matrix}] := [\begin{matrix}tr(ρ(X)) \\ tr(ρ(Y)) \\ tr(ρ(XY)) \end{matrix}] .

In terms of these coordinates, the trace tr ρ(K) equals:

κ(x, y, z) := x^{2} + y^{2} + z^{2} − xyz − 2

which is preserved under the action of Out(π). The action of Out(π) on ℂ^{3} is commensurable with the action of the group Γ of polynomial automorphisms of ℂ^{3} which preserve κ (Horowitz [17]). (Compare also Magnus [22].)

Theorem. Let κ(x, y, z) = x^{2} + y^{2} + z^{2} − xyz − 2 and let t ∈ ℝ. • For t < −2, the group Γ acts properly on κ^{−1}(t) ∩ ℝ^{3};

• For −2 ≤ t < 2, there is a compact connected component C_{t} of κ^{−1}(t) ∩ ℝ^{3} upon which Γ acts ergodically; Γ acts properly on the complement (κ^{−1}(t) ∩ ℝ^{3}) − C_{t};

• For t = 2, the action of Γ is ergodic on the compact subset κ^{−1}(2) ∩ [−2, 2]^{3} and the action is ergodic on the complement κ^{−1}(2) − [−2, 2]^{3};

• For 18 ≥ t > 2, the group Γ acts ergodically on κ^{−1}(t) ∩ ℝ^{3};

• For t > 18, the group Γ acts properly and freely on an open subset Ω_{t} ⊂ κ^{−1}(t) ∩ ℝ^{3}, permuting its components.The Γ-action on the complement of Ω_{t} is ergodic.

The proof uses the interplay between representations of the fundamental group and hyperbolic structures on M. The dynamics breaks up into two strikingly different types: representations corresponding to hyperbolic structures comprise contractible connected components of the the level sets κ^{−1}(t), whereas representations which map a simple nonperipheral essential loop to an elliptic element, comprise open subsets of κ^{−1}(t) upon which Γ is ergodic. Thus nontrivial dynamics accompanies nontrivial topology of the moduli spaces.

In his doctoral thesis [31], G. Stantchev considers characters corresponding to representations into PGL(2, ℝ) (that is, actions preserving H^{2} but not preserving orientation on H^{2}). For t < −14 (respectively t > 6) characters of discrete embeddings representing hyperbolic structures on 2-holed projective planes (respectively 1-holed Klein bottles) give wandering domains in the corresponding level set. For −14 ≤ t < 2 the Γ-action on the corresponding level set is ergodic, and for t < −14, the action is ergodic on the complement of the wandering domains corresponding to Fricke spaces of 2-holed projective planes.

4 WILLIAM M. GOLDMAN

Figure 1. Level set κ = −2.1 Figure 2. Level set κ = 1.9

ACTION OF THE MODULAR GROUP 5

Figure 3. Level set κ = 1.9 Figure 4. Level set κ = 2.1

6 WILLIAM M. GOLDMAN

Figure 5. Level set κ = 2.1 Figure 6. Level set κ = 4

ACTION OF THE MODULAR GROUP 7

Figure 7. Level set κ = 4 Figure 8. Level set κ = 19 Notation and terminology. We work in Poincaré’s model of the hyperbolic plane H^{2} as the upper half-plane. We denote the identity map (identity matrix) by 𝕀. Commutators are denoted:

[A, B] = ABA^{−1}B^{−1}

and inner automorphisms are denoted:

ι_{g} : x ⟼ gxg^{−1}.

For any group π we denote the group of all automorphisms by Aut(π) and the normal subgroup of inner automorphisms by Inn(π). We denote the quotient group Aut(π)/ Inn(π) by Out(π). A compact surface S with n boundary components will be called n-holed. Thus a one-holed torus is the complement of an open disc inside a torus and a three-holed sphere (sometimes called a “pair-of-pants”) is the complement of three disjoint discs inside a sphere.

Acknowledgements. I am grateful to Matt Bainbridge, Robert Benedetto, Lawrence Ein, Cathy Jones, Misha Kapovich, Bernhard Leeb, John Millson, John Parker, Dan Rudolph, Peter Shalen, Richard Schwartz, Adam Sikora, and Scott Wolpert for helpful conversations during the course of this work. I especially want to thank Walter Neumann and George Stantchev for carefully reading the manuscript and suggesting major improvements.

1. The modular group and the moduli space

In this section we define the modular group and the moduli space for the one-holed torus. The modular group is isomorphic to

π_{0}(Homeo(M)) ≅ Out(π) ≅ GL(2, ℤ)

and the moduli space is affine space ℂ^{3}. We explain how the invariant function κ originates. We describe various elements of the modular group and how they act on the moduli space.

1.1. The mapping class group. Let M denote a compact connected orientable surface of genus one with one boundary component. Since attaching a disc to M yields a torus, we refer to M as a one-holed torus. The mapping class group of M is the group π_{0}(Homeo(M, ∂M)) of isotopy classes of homeomorphisms of M. We investigate the action of π_{0}(Homeo(M, ∂M)) on the moduli space of flat SL(2)-connections on M.

1.1.1. Relation to π_{1}(M). Choose a basepoint x_{0} ∈ M and let π := π_{1}(M; x_{0}). Any homeomorphism of M is isotopic to one which fixes x_{0} and hence defines an automorphism of π. Two such isotopic homeomorphisms determine automorphisms of π differing by an inner automorphism, producing a well-defined homomorphism

(1.1.1)  N : π_{0}(Homeo(M)) ⟶ Out(π) ≅ Aut(π)/ Inn(π).

If M is a closed surface, then Dehn (unpublished) and Nielsen [30] proved that N is an isomorphism. (See Stillwell [32] for a proof of the Dehn-Nielsen theorem.) When ∂M ≠ ∅, then each component ∂_{i}M determines a conjugacy class C_{i} of elements of π_{1}(M). The image of N consists of elements of Out(π) represented by automorphisms which preserve each C_{i}.

Let M be a one-holed torus. Its fundamental group admits a geometric redundant presentation

(1.1.2)  π = ⟨X, Y, K | [X, Y] = K⟩

where K corresponds to the generator of π_{1}(∂M). Of course, π is freely generated by X, Y.

1.1.2. Nielsen’s theorem. The following remarkable property of M is due to Jakob Nielsen [29] and does not generalize to other hyperbolic surfaces with boundary. For a proof see Magnus-Karrass-Solitar [23], Theorem 3.9 or Lyndon-Schupp [21], Proposition 5.1.

Proposition 1.1.1. Any automorphism of the rank two group

π = ⟨X, Y, K | K = [X, Y]⟩

takes K to a conjugate of either K itself or its inverse K^{−1}.

An equivalent geometric formulation is:

Proposition 1.1.2. Every homotopy-equivalence M → M is homotopic to a homeomorphism of M.

Thus the homomorphism (1.1.1) defines an isomorphism

N : π_{0}(Homeo(M)) ⟶ Out(π).

1.2. The structure of the modular group. We say that an automorphism of π which takes K to either K or K^{−1} is normalized. The normalized automorphisms form a subgroup Aut(π, K) of Aut(π).

Let ϕ be an automorphism of π. Proposition 1.1.1 implies that ϕ(K) is conjugate to K^{±1}. Thus there exists an inner automorphism ι_{g} such that

ϕ(K) = gK^{ϵ}g^{−1},

that is

ι_{g}^{−1} ∘ ϕ ∈ Aut(π, K)

where ϵ = ±1. (In fact, ϵ = det(h([ϕ])), where h is the homomorphism defined by (1.2.1) below.) Since the centralizer of K in π equals the cyclic group ⟨K⟩, the automorphism ϕ determines the coset of g modulo ⟨K⟩ uniquely.

We obtain a short exact sequence

1 ⟶ ⟨ι_{K}⟩ ↪ Aut(π, K) ⟶ Out(π) ⟶ 1

The action on the homology H_{1}(M; ℤ) ≅ ℤ^{2} defines a homomorphism

(1.2.1)  h : Out(π) ⟶ GL(2, ℤ).

By Nielsen [29], h is an isomorphism. (Surjectivity follows by realizing an element of GL(2, ℤ) as a linear homeomorphism of the torus ℝ^{2}/ℤ^{2}. See Lyndon-Schupp [21], Prop. 4.5 or Magnus-Karrass-Solitar [23], §3.5, Corollary N4.) We obtain an isomorphism

μ : GL(2, ℤ) ⟶ Aut(π, K)/⟨ι_{K}⟩.

Restriction of the composition μ ∘ h to Aut(π, K) equals the quotient homomorphism

Aut(π, K) ⟶ Aut(π, K)/⟨ι_{K}⟩.

2. Structure of the character variety

2.1. Trace functions. The relevant moduli space is the character variety, the categorical quotient of Hom(π, G) by the G-action by inner automorphisms, where G = SL(2, ℂ). Since π is freely generated by two elements X and Y, the set Hom(π, G) of homomorphisms π ⟶ G identifies with the set of pairs (ξ, η) ∈ G × G, via the mapping

Hom(π, G) ⟶ G × G
ρ ⟼ (ρ(X), ρ(Y)).

This mapping is equivariant respecting the action of G on Hom(π, G) by

g : ρ ⟼ ι_{g} ∘ ρ

and the action of G on G × G by

g : (ξ, η) ⟼ (gξg^{−1}, gηg^{−1}).

The moduli space Hom(π, G)//G consists of equivalence classes of elements of Hom(π, G) ≅ G × G where the equivalence class of a homomorphism ρ is defined as the closure of the G-orbit Gρ. Then Hom(π, G)//G is the categorical quotient in the sense that its coordinate ring identifies with the ring of G-invariant regular functions on Hom(π, G). For a single element g ∈ G, the conjugacy class Inn(G)(g) is determined by the trace t = tr(g) if t ≠ ±2. That is,

(2.1.1)  Inn(G)(g) = tr^{−1}(t).

For t = ±2, then

(2.1.2)  tr^{−1}(±2) = \{±𝕀\} ⋃ Inn(G) · (± [\begin{matrix}1 & 1 \\ 0 & 1 \end{matrix}]) .

By Fricke [8] and Fricke-Klein [9], the traces of the generators X, Y, XY parametrize Hom(π, G)//G as the affine space ℂ^{3}. As X and Y freely generate π, we may identify:

Hom(π, G) ⟷ G × G
ρ ⟷ (ρ(X), ρ(Y))

The character mapping

χ : Hom(π, G)//G ⟶ ℂ^{3}
[ρ] ⟼ [\begin{matrix}x(ρ) \\ y(ρ) \\ z(ρ) \end{matrix}] = [\begin{matrix}tr(ρ(X)) \\ tr(ρ(Y)) \\ tr(ρ(XY)) \end{matrix}]

is an isomorphism. (Compare the discussion in Goldman [12], 4.1, [13], §§4–5 and [14].)

For example, given (x, y, z) ∈ ℂ^{3}, the representation ρ defined by

(2.1.3)  ρ(X) = [\begin{matrix}x & −1 \\ 1 & 0 \end{matrix}] , ρ(Y) = [\begin{matrix}0 & ζ^{−1} \\ −ζ & y \end{matrix}]

satisfies χ(ρ) = (x, y, z) where ζ ∈ ℂ is chosen so that

ζ + ζ^{−1} = z.

Conversely, if (x, y, z) ∈ ℂ^{3} then χ^{−1}(x, y, z) consists of a single G-orbit if and only if κ(x, y, z) ≠ 2 where κ is defined below in (2.1.4). (This is also the condition that ρ is an irreducible representation of π. Compare Lubotzky-Magid [20], Brumfiel-Hilden [4], Culler-Shalen [6].)

For any word w(X, Y), the function

Hom(π, G) ⟶ ℂ
[ρ] ⟼ tr(ρ(w(X, Y))

is G-invariant. Hence there exists a polynomial f_{w}(x, y, z) ∈ ℂ[x, y, z] such that

tr(ρ(w(X, Y)) = f_{w}(x(ρ), y(ρ), z(ρ)).

A particularly important example occurs for w(X, Y) = [X, Y] = K, in which case we denote f_{w}(x, y, z) by κ(x, y, z). By an elementary calculation (see, for example [14]),

(2.1.4)  tr(ρ(K)) = κ(x, y, z) := x^{2} + y^{2} + z^{2} − xyz − 2.

The level set κ^{−1}(t) consists of equivalence classes of representations ρ : π → G where ρ(K) is constrained to lie in tr^{−1}(t). (Compare (2.1.1) and (2.1.2) above.)

2.2. Automorphisms. Let G = SL(2, ℂ). The group Aut(π) acts on the character variety Hom(π, G)//G by:

ϕ_{∗}([ρ]) = [ρ ∘ ϕ^{−1}]

and since

ρ ∘ ι_{γ} = ι_{ρ(γ)} ∘ ρ

the subgroup Inn(π) acts trivially. Thus Out(π) acts on

Hom(π, G)//G ≅ ℂ^{3}

and since an automorphism ϕ of π is determined by

(ϕ(X), ϕ(Y)) = (w_{1}(X, Y), w_{2}(X, Y)),

the action of ϕ on ℂ^{3} is given by the three polynomials f_{w_{1}}, f_{w_{2}}, f_{w_{3}}:

[\begin{matrix}x \\ y \\ z \end{matrix}] ⟼ [\begin{matrix}f_{w_{1}}(x, y, z) \\ f_{w_{2}}(x, y, z) \\ f_{w_{3}}(x, y, z) \end{matrix}] .

where w_{3}(X, Y) = w_{1}(X, Y)w_{2}(X, Y) is the product word. Hence Out(π) acts on ℂ^{3} by polynomial automorphisms. Nielsen’s theorem (Proposition 1.1.1) implies that any such automorphism preserves κ : ℂ^{3} ⟶ ℂ, that is

κ (f_{w_{1}}(x, y, z), f_{w_{2}}(x, y, z), f_{w_{1}w_{2}}(x, y, z)) = κ(x, y, z).

2.2.1. Sign-change automorphisms. Some automorphisms of the character variety are not induced by automorphisms of π. Namely, the homomorphisms of π into the center {±𝕀} ⊂ G form a group acting on Hom(π, G) by pointwise multiplication. Let ζ ∈ Hom(π, {±𝕀}) and ρ ∈ Hom(π, G). Then

(2.2.1)  ζ · ρ : γ ⟼ ζ(γ)ρ(γ)

is a homomorphism. This defines an action

Hom(π, {±𝕀}) × Hom(π, G) ⟶ Hom(π, G).

Furthermore, since {±𝕀} is central in G and K ∈ π is a commutator

(2.2.2)  (ζ · ρ)(K) = ρ(K).

Since π is free of rank two,

Hom(π, {±𝕀}) ≅ ℤ/2 × ℤ/2.

The three nontrivial elements (0, 1), (1, 0), (1, 1) of ℤ/2 × ℤ/2 act on representations by σ_{1}, σ_{2}, σ_{3} respectively:

σ_{1} · ρ : \begin{cases}X & ⟼ ρ(X) \\ Y & ⟼ −ρ(Y) \end{cases}

σ_{2} · ρ : \begin{cases}X & ⟼ −ρ(X) \\ Y & ⟼ ρ(Y) \end{cases}

σ_{3} · ρ : \begin{cases}X & ⟼ −ρ(X) \\ Y & ⟼ −ρ(Y) \end{cases}

The corresponding action on characters is:

(σ_{1})_{∗} : [\begin{matrix}x \\ y \\ z \end{matrix}] ⟼ [\begin{matrix}x \\ −y \\ −z \end{matrix}]

(σ_{2})_{∗} : [\begin{matrix}x \\ y \\ z \end{matrix}] ⟼ [\begin{matrix}−x \\ y \\ −z \end{matrix}]

(σ_{3})_{∗} : [\begin{matrix}x \\ y \\ z \end{matrix}] ⟼ [\begin{matrix}−x \\ −y \\ z \end{matrix}] .

We call this group the group of sign-changes and denote it by Σ. Evidently Σ preserves κ(x, y, z). (Alternatively apply (2.2.2).)

2.2.2. Permutations. Since κ(x, y, z) is symmetric in x, y, z the full symmetric group 𝔖_{3} also acts on ℂ^{3} preserving κ. Unlike Σ, elements of 𝔖_{3} are induced by automorphisms of π. The group of all linear automorphisms of (ℂ^{3}, κ) is generated by Σ and 𝔖_{3} and forms a semidirect product Σ ⋊ 𝔖_{3}.

𝔖_{3} is actually a quotient of Γ. The projective line ℙ^{1}(ℤ/2) over ℤ/2 has three elements, and every permutation of this set is realized by a projective transformation. Since ℤ/2 has only one nonzero element, the projective automorphism group of ℙ^{1}(ℤ/2) equals GL(2, ℤ/2) and GL(2, ℤ/2) ≅ 𝔖_{3}. The action of GL(2, ℤ) on (ℤ/2)^{2} defines a homomorphism

GL(2, ℤ) ⟶ GL(2, ℤ/2) ≅ 𝔖_{3}

whose kernel GL(2, ℤ)_{(2)} is generated by the involutions

(2.2.3)  [\begin{matrix}1 & 0 \\ 0 & −1 \end{matrix}] , [\begin{matrix}1 & −2 \\ 0 & −1 \end{matrix}] , [\begin{matrix}1 & 0 \\ 2 & −1 \end{matrix}]

and −𝕀 ∈ GL(2, ℤ)_{(2)}. The sequence

PGL(2, ℤ)_{(2)} := GL(2, ℤ)_{(2)}/{±𝕀} ⟶ PGL(2, ℤ) ⟶ 𝔖_{3}

is exact. The kernel Γ_{(2)} of the composition

Γ ⟶ PGL(2, ℤ) ⟶ 𝔖_{3},

equals the semidirect product

PGL(2, ℤ)_{(2)} ⋉ (ℤ/2 ⊕ ℤ/2)

and Γ is an extension

Γ_{(2)} ⟶ Γ ⟶ 𝔖_{3}.

(See Goldman-Neumann [15] for more extensive discussion of Γ and its action on the the set of complex points of the character variety.)

2.2.3. Other automorphisms. For later use, here are several specific elements of Γ.

The elliptic involution is the automorphism

X ⟼ X^{−1}
Y ⟼ Y^{−1}

corresponding to −𝕀 ∈ GL(2, ℤ) and acts trivially on characters. The Dehn twist about X is the automorphism τ_{X} ∈ Aut(π)

X ⟼ X
Y ⟼ Y X

inducing the automorphism of characters

[\begin{matrix}x \\ y \\ z \end{matrix}] ⟼ [\begin{matrix}x \\ xy − z \\ y \end{matrix}]

and corresponds to

[\begin{matrix}1 & 1 \\ 0 & 1 \end{matrix}] ∈ GL(2, ℤ).

The Dehn twist about Y is the automorphism τ_{Y} ∈ Aut(π)

X ⟼ XY
Y ⟼ Y

inducing the automorphism of characters

[\begin{matrix}x \\ y \\ z \end{matrix}] ⟼ [\begin{matrix}xy − z \\ y \\ x \end{matrix}] .

and corresponds to

[\begin{matrix}1 & 0 \\ 1 & 1 \end{matrix}] ∈ GL(2, ℤ).

The quadratic reflection Q_{z} is the automorphism

X ⟼ X
Y ⟼ Y^{−1}

inducing the automorphism of characters

[\begin{matrix}x \\ y \\ z \end{matrix}] ⟼ [\begin{matrix}x \\ y \\ xy − z \end{matrix}] .

and corresponds to

[\begin{matrix}1 & 0 \\ 0 & −1 \end{matrix}] ∈ GL(2, ℤ).

2.3. Reducible characters. Any representation ρ having character in κ^{−1}(2) is reducible, as can be checked by direct calculation of commutators in SL(2, ℂ). Namely, let ξ, η ∈ SL(2, ℂ). We may write

η = [\begin{matrix}a & b \\ c & d \end{matrix}]

where ad − bc = 1. By applying an inner automorphism, we may assume that ξ is in Jordan canonical form. If ξ is diagonal,

ξ = [\begin{matrix}λ & 0 \\ 0 & λ^{−1} \end{matrix}]

then

tr[ξ, η] = 2 + bc(λ − λ^{−1})^{2}

implies that if tr[ξ, η] = 2, then either ξ = ±𝕀 or bc = 0 (so ξ is upper-triangular or lower-triangular). Otherwise

ξ = ± [\begin{matrix}1 & s \\ 0 & 1 \end{matrix}] ,

(where s ≠ 0), in which case

(2.3.1)  tr[ξ, η] = 2 + s^{2}c^{2} ≥ 2

so tr[ξ, η] = 2 implies that c = 0 and η is upper-triangular. Thus if tr(ρ(K)) = 2, then ρ is conjugate to an upper-triangular representation.

We may replace ρ by its semisimplification, that is the upper-triangular matrices by the corresponding diagonal matrices (their semisimple parts) to obtain a representation by diagonal matrices having the same character:

(2.3.2)  ρ(X) = [\begin{matrix}ξ & ∗ \\ 0 & ξ^{−1} \end{matrix}]
ρ(Y) = [\begin{matrix}η & ∗ \\ 0 & η^{−1} \end{matrix}]
ρ(XY) = [\begin{matrix}ζ & ∗ \\ 0 & ζ^{−1} \end{matrix}] .

where ξη = ζ. Thus

(2.3.3)  x = ξ + ξ^{−1},
y = η + η^{−1},
z = ζ + ζ^{−1}.

(2.3.3) corresponds to the following factorization (2.3.4) of

κ(x, y, z) − 2 = x^{2} + y^{2} + z^{2} − xyz − 4.

Under the embedding

ℂ[x, y, z] ⟶ ℂ[ξ, ξ^{−1}, η, η^{−1}, ζ, ζ^{−1}]

defined by (2.3.3), the polynomial κ(x, y, z) − 2 factors:

(2.3.4)  κ(ξ + ξ^{−1}, η + η^{−1}, ζ + ζ^{−1}) − 2 =
ζ^{−2}(1 − ξηζ)(1 − ξ^{−1}ηζ)(1 − ξη^{−1}ζ)(1 − ξηζ^{−1}).

Given (x, y, z) with κ(x, y, z) = 2, the triple (ξ, η, ζ) is only defined up to an action of ℤ/2 ⊕ ℤ/2. Namely the automorphisms

\begin{cases}X & ⟼ X^{−1} \\ Y & ⟼ Y \end{cases}

\begin{cases}X & ⟼ X \\ Y & ⟼ Y^{−1} \end{cases}

\begin{cases}X & ⟼ X^{−1} \\ Y & ⟼ Y^{−1} \end{cases}

act on (ξ, η, ζ) by:

\begin{cases}ξ & ⟼ ξ^{−1} \\ η & ⟼ η \\ ζ & ⟼ ζ^{−1} \end{cases}

\begin{cases}ξ & ⟼ ξ \\ η & ⟼ η^{−1} \\ ζ & ⟼ ζ^{−1} \end{cases}

\begin{cases}ξ & ⟼ ξ^{−1} \\ η & ⟼ η^{−1} \\ ζ & ⟼ ζ \end{cases}

respectively.

2.4. The Poisson structure. As in Goldman [13],§5.3, the automorphisms preserving κ are unimodular and therefore preserve the exterior bivector field as well

Ξ = \frac{1}{2π^{2}} dκ · \frac{∂}{∂x} ∧ \frac{∂}{∂y} ∧ \frac{∂}{∂z}

= \frac{1}{2π^{2}} ( (2x − yz)\frac{∂}{∂y} ∧ \frac{∂}{∂z}

+ (2y − zx)\frac{∂}{∂z} ∧ \frac{∂}{∂x}

+ (2z − xy)\frac{∂}{∂x} ∧ \frac{∂}{∂y} )

which restricts to (the dual of) an area form on each level set κ^{−1}(t) which is invariant under Out(π). We shall always consider this invariant measure on κ^{−1}(t) ∩ ℝ^{3}. In the case when κ^{−1}(t) has a rational parametrization by an affine plane, that is, when t = 2, the above bivector field is the image of the constant bivector field under the parametrization:

ℂ × ℂ ⟶ κ^{−1}(2)
(ξ, η) ⟼ [\begin{matrix}e^{ξ} + e^{−ξ} \\ e^{η} + e^{−η} \\ e^{ξ+η} + e^{−(ξ+η)} \end{matrix}]

\frac{∂}{∂ξ} ∧ \frac{∂}{∂η} ⟼ 2π^{2}Ξ

The bivector field Ξ defines a Poisson structure on the moduli space ℂ^{3} for which κ defines a Casimir function.

(The coefficient 2π^{2} occurs to normalize the area of the compact component C_{K} of reducible SU(2)-characters to 1.) Here we study the action of Out(π) on the set κ^{−1}(t) ∩ ℝ^{3} of ℝ-points of κ^{−1}(t) where t ∈ ℝ, with respect to this invariant measure.

2.5. The orthogonal representation. To describe the structure of the character variety more completely, we use a 3-dimensional orthogonal representation associated to a character (x, y, z) ∈ ℂ^{3}. (Compare §4.2 of Goldman [12] and Brumfiel-Hilden [4].) This will be used to identify real characters as characters of representations into the real forms SL(2, ℝ) and SU(2) of SL(2, ℂ).

Consider the complex vector space ℂ^{3} with the standard basis {e_{1}, e_{2}, e_{3}} and bilinear form defined by the symmetric matrix

𝔹 = [\begin{matrix}2 & z & y \\ z & 2 & x \\ y & x & 2 \end{matrix}] .

Since

det(𝔹) = −2(κ(x, y, z) − 2)

(where κ(x, y, z) is defined in (2.1.4)), the symmetric bilinear form is nondegenerate if and only if κ(x, y, z) ≠ 2.

Assume κ(x, y, z) ≠ 2 so that 𝔹 is nondegenerate. Let SO(ℂ^{3}, 𝔹) ≅ SO (3, ℂ) denote the group of unimodular linear transformations of ℂ^{3} orthogonal with respect to 𝔹. The local isomorphism

Φ : SL(2, ℂ) ⟶ SO(ℂ^{3}, 𝔹).

is a surjective double covering, equivalent to the adjoint representation (or the representation on the second symmetric power of ℂ^{2}), and is unique up to composition with automorphisms of SL(2, ℂ) and SO (3, ℂ).

For any vector v ∈ ℂ^{3} such that 𝔹(v, v) ≠ 0, the reflection

(2.5.1)  R_{v} : x ⟼ x − 2 \frac{𝔹(v, x)}{𝔹(v, v)}v

is a 𝔹-orthogonal involution.

Let ℤ/2 ⋆ ℤ/2 ⋆ ℤ/2 be the free product of cyclic groups ⟨R_{i}⟩, where i = 1, 2, 3, and R_{i}^{2} = 𝕀. The homomorphism

π ⟶ ℤ/2 ⋆ ℤ/2 ⋆ ℤ/2
X ⟼ R_{2}R_{3}
Y ⟼ R_{3}R_{1}

embeds π as an index two subgroup of ℤ/2 ⋆ ℤ/2 ⋆ ℤ/2.

Lemma 2.5.1. Let ρ ∈ Hom(π, G) satisfy κ(x, y, z) ≠ 2. The restriction to π of the representation ρ̂: ℤ/2⋆ℤ/2⋆ℤ/2 ⟶ SO(ℂ^{3}, 𝔹) defined by R_{i} ⟼ R_{e_{i}} is conjugate to the composition Φ ∘ ρ : π ⟶ SO(ℂ^{3}, 𝔹).

See §4.2 of Goldman [12], or Brumfiel-Hilden [4] for a proof.

2.6. The set of ℝ-points. A real character is the character of a representation into a real form of G. (Proposition III.1.1 of Morgan-Shalen [26]). The above orthogonal representation gives an explicit form of this result.

Suppose (x, y, z) ∈ ℝ^{3} and κ(x, y, z) ≠ 2. Then the restriction of 𝔹 to ℝ^{3} is a nondegenerate symmetric ℝ-valued bilinear form (also denoted 𝔹), which is either indefinite or (positive) definite. Let SO(ℝ^{3}, 𝔹) denote the group of unimodular linear transformations of ℝ^{3} orthogonal with respect to 𝔹.

There are two conjugacy classes of real forms of G, compact and noncompact. Every compact real form of G is conjugate to SU(2) and every noncompact real form of G is conjugate to SL(2, ℝ). Specifically, if 𝔹 is positive definite, then SO(ℝ^{3}, 𝔹) is conjugate to SO(3) and Φ^{−1}(SO(ℝ^{3}, 𝔹)) is conjugate to SU(2). If 𝔹 is indefinite, then SO(ℝ^{3}, 𝔹) is conjugate to either SO(2, 1) or SO(1, 2) (depending on whether κ(x, y, z) > 2 or κ(x, y, z) < 2 respectively) and Φ^{−1}(SO(ℝ^{3}, 𝔹)) is conjugate to SL(2, ℝ). Thus every real character (x, y, z) ∈ ℝ^{3} is the character of a representation into either SU(2) or SL(2, ℝ).

When x, y, z ∈ ℝ, the restriction of 𝔹 to ℝ^{3} is definite if and only if −2 ≤ x, y, z ≤ 2 and κ(x, y, z) < 2.

The coordinates of characters of representations π_{1}(M) ⟶ SU(2) satisfy:

−2 ≤ x, y, z ≤ 2, x^{2} + y^{2} + z^{2} − xyz − 2 ≤ 2

and, for fixed t ∈ [−2, 2], comprise a component of κ^{−1}(t) ∩ ℝ^{3}. The other four connected components of κ^{−1}(t) ∩ ℝ^{3} consist of characters of SL(2, ℝ)-representations. These four components are freely permuted by Σ. For t < −2, all four components of the relative character variety κ^{−1}(t) ∩ ℝ^{3} consist of characters of SL(2, ℝ)-representations and are freely permuted by Σ. For t > 2, the relative character variety κ^{−1}(t) ∩ ℝ^{3} is connected and consists of characters of SL(2, ℝ)- representations.

The two critical values ±2 of tr : G ⟶ ℂ deserve special attention. When t = −2, a representation ρ ∈ Hom(π, G) with tr ρ(K) = −2 is a regular point of the mapping

E_{K} : Hom(π, G) ⟶ G
ρ  ⟼ ρ(K).

Such a representation is a regular point of the composition

tr ∘E_{K} : Hom(π, G) ⟶ ℂ

unless ρ(K) = −I. In the latter case (when [ρ(X), ρ(Y)] = −I), the representation is conjugate to the quaternion representation in SU(2)

(2.6.1)  ρ(X) = [\begin{matrix}i & 0 \\ 0 & −i \end{matrix}] , ρ(Y) = [\begin{matrix}0 & −1 \\ 1 & 0 \end{matrix}] .

In both cases, G acts locally freely on the subset (tr ∘E_{K})^{−1}(−2) of Hom(π, G) with quotient κ^{−1}(−2).

Now consider the case t = 2. As in §2.3, the ℝ-points of κ^{−1}(2) correspond to reducible representations, and in fact are characters of representations with values in the Cartan subgroups (maximal tori) of the real forms SU(2) and SL(2, ℝ). Every Cartan subgroup of SU(2) is conjugate to U(1), and every Cartan subgroup of SL(2, ℝ) is conjugate to either SO (2) or SO (1, 1). Characters of reducible SU(2)- representations form the compact set

C_{K} = κ^{−1}(2) ∩ [−2, 2]^{3}

which identifies with the quotient of the 2-torus U(1) × U(1) by {±𝕀} under the extension (2.3.3).

Characters of reducible SL(2, ℝ)-representations comprise four components (related by C_{i} = σ_{i}C_{0} for i = 1, 2, 3):

C_{0} = κ^{−1}(2) ∩ ([2, ∞) × [2, ∞) × [2, ∞)),
C_{1} = κ^{−1}(2) ∩ ([2, ∞) × (−∞, −2] × (−∞, −2]),
C_{2} = κ^{−1}(2) ∩ ((−∞, −2] × [2, ∞) × (−∞, −2]),
C_{3} = κ^{−1}(2) ∩ ((−∞, −2] × (−∞, −2] × [2, ∞)),

each of which identifies with the quotient of ℝ_{+} × ℝ_{+} by {±𝕀} under (2.3.3). These four components are freely permuted by Σ. There is a compact component of equivalence classes of SL(2, ℝ)-representations which are irreducible but not absolutely irreducible — that is, although ℝ^{2} is an irreducible π-module, its complexification is reducible. In that case the representation is conjugate to a representation in SO (2). This component agrees with the component C_{K} consisting of characters of reducible SU(2)-representations.

3. Hyperbolic structures on tori (t < 2)

The topology and dynamics change dramatically as t changes from t < −2 to t > 2. The level sets t < −2 correspond to Fricke spaces of one-holed tori with geodesic boundary. The level set for t = −2 consists of four copies the Teichmüller space of the punctured torus, together with {(0, 0, 0)}. (The origin is the character of the quaternion representation in SU(2).) The origin is fixed under the action, while Γ acts properly on its complement in κ^{−1}(−2). (However the action on the set of complex points of κ^{−1}(−2) is extremely nontrivial and mysterious; see Bowditch [3].)

The level sets for −2 < t < 2 correspond to Teichmüller spaces of singular hyperbolic structures with one singularity, as well as a (compact) component consisting of characters of unitary representations. Except for the component of unitary representations, there are four components, freely permuted by Σ. Except for the component of unitary representations, the Γ-action is proper.

3.1. Complete hyperbolic structures (t ≤ −2). The theory of deformations of geometric structures implies that Γ acts properly on certain components of κ^{−1}(t). When t < 2, the moduli space κ^{−1}(t) contains 4 contractible noncompact components, freely permuted by Σ.

(When −2 ≤ t < 2, an additional compact component corresponds to SU(2)-representations.) These contractible components correspond to SL(2, ℝ)-representations, and each one identifies with the Teichmüller space of M, with certain boundary conditions. Specifically, if t < −2, then these components correspond to hyperbolic structures on int(M) with geodesic boundary of length 2 cosh^{−1}(−t/2). For t = −2, these components correspond to complete hyperbolic structures on int(M) and identify with the usual Teichmüller space 𝔗_{M}. For −2 < t < 2, these components correspond to singular hyperbolic structures on a torus whose singularity is an isolated point with cone angle

θ = 2 cos^{−1}(−t/2).

3.2. Complete structures and proper actions. We begin with the case t < −2. Then each component of κ^{−1}(t) ∩ ℝ^{3} parametrizes complete hyperbolic structures on int(M) with a closed geodesic parallel to ∂M having length 2 cosh^{−1}(−t/2). Thus the union

⋃_{t≤−2} κ^{−1}(t)

consists of four copies of the Fricke space of M. This space contains equivalence classes of marked complete hyperbolic structures on int(M), not necessarily of finite area. The properness of the action of the mapping class group π_{0}(Homeo(M)) on the Fricke space of M implies properness of the Γ-action of κ^{−1}(t). (For proof, see §2.2 of Abikoff [1], Bers-Gardiner [2] 6.5.6 (p.156) of Buser [5], 6.3 of Imayoshi-Tanigawa [19], 2.4.1 of Harvey [16], or §2.7 of Nag [28].)

When t = −2, then κ^{−1}(t) has five connected components. It is the union of the single (singular) point (0, 0, 0) and four copies of the Teichmüller space 𝔗_{M} of M. Teichmüller space 𝔗_{M} consists of equivalence classes of marked complete finite-area hyperbolic structures on M. The group Γ acts properly on κ^{−1}(t) and fixes (0, 0, 0).

3.3. Singular hyperbolic structures on tori (−2 < t < 2). We next consider the case 2 > t > −2. Here κ^{−1}(t) has five connected components. One component C_{t} is compact and consists of unitary characters, while the other four components correspond to singular hyperbolic structures on a torus with a cone point. C_{t} is diffeomorphic to S^{2}, and the symplectic structure is a smooth area form. Γ acts ergodically on C_{t}. See §5 of Goldman [13] for a detailed discussion.

Consider next the four noncompact components of κ^{−1}(t). This case is similar to the previous case, except that the ends of M are replaced by cone points on a torus. Using the properness of the mapping class group action on Teichmüller space, the action on these components of the relative character variety remains proper. However, none of the corresponding representations in Hom(π, G) are discrete embeddings. Generically these representations are isomorphisms onto dense subgroups of SL(2, ℝ).

We first show (Theorem 3.4.1) that every representation in these components is a lift (to SL(2, ℝ)) of the holonomy representation of a singular hyperbolic structure. §3.5 proves that In §3.5), we deduce that Γ acts properly on the level sets κ^{−1}(t) where −2 < t < 2.

3.4. Construction of hyperbolic structures on T^{2} with one cone point. Let θ > 0 and let ℭ_{θ} denote the space of hyperbolic structures on T^{2} with a conical singularity of cone angle θ. By results of McOwen [25] and Troyanov [33], ℭ_{θ} identifies with the Teichmüller space 𝔗(θ), the deformation space of conformal structures on T^{2} singular at one point with cone angle θ.

Theorem 3.4.1. Suppose ξ, η ∈ SL(2, ℝ) and γ := [ξ, η] is elliptic of rotation angle θ. Let M denote a one-holed torus whose fundamental group has the presentation (1.1.2). Then there exists a singular hyperbolic structure on M̄ with a singularity of cone angle θ having holonomy representation ρ defined by

ρ(X) = ξ, ρ(Y) = η, ρ(K) = γ.

The proof will be based on the following.

Lemma 3.4.2. Let p ∈ H^{2} be a point fixed by γ and consider the points

p_{4} = p,
p_{3} = η^{−1}p,
p_{2} = ξ^{−1}η^{−1}p,
p_{1} = ηξ^{−1}η^{−1}p.

The four points p_{1}, p_{2}, p_{3}, p_{4} are the vertices of an embedded quadrilateral Q. In other words the four segments

l_{1} = \overline{p_{1}p_{2}}
l_{2} = \overline{p_{2}p_{3}}
l_{3} = \overline{p_{3}p_{4}}
l_{4} = \overline{p_{4}p_{1}}

are disjoint and bound a quadrilateral.

Proof of Theorem 3.4.1 assuming Lemma 3.4.2.

ξ maps p_{1} (respectively p_{2}) to p_{4} (respectively p_{3}). Therefore ξ maps the directed edge l_{1} to l_{3} with the opposite orientation. Similarly η maps p_{2} (respectively p_{3}) to p_{1} (respectively p_{4}) so η maps l_{2} to l_{4} with the opposite orientation.

From the embedding of Q and the identifications of its opposite sides by ξ and η, we construct a developing map for a (singular) hyperbolic structure on M as follows. Let Q denote an abstract quadrilateral, that is, a cell complex with a single 2-cell, four 1-cells (the sides), and four 0-cells (the vertices). Denote the oriented edges L_{i} numbered in cyclic order and the vertices p_{i} (i = 1, 2, 3, 4) where

∂L_{1} = {p_{1}, p_{2}}, ∂L_{2} = {p_{2}, p_{3}}
∂L_{3} = {p_{3}, p_{4}}, ∂L_{4} = {p_{4}, p_{1}}.

Let L̄_{i} denote L_{i} with the opposite orientation. Write

\begin{matrix}.... \\ Q \end{matrix} = Q − \{p_{1}, p_{2}, p_{3}, p_{4}\}.

Choose orientation-preserving homeomorphisms

ξ̃: L_{1} ⟶ L̄_{3},  η̃: L_{2} ⟶ L̄_{4}.

Let Π = ⟨X, Y ⟩ denote the free group generated by X, Y. A model for the universal covering M̃ of M is the quotient space of the Cartesian product

\begin{matrix}.... \\ Q \end{matrix} × Π

by the equivalence relation generated by the identifications

(x, η) ⟷ (ξ̃(x), Xη)

for x ∈ L_{1}, and

(x, η) ⟷ (η̃(x), Y η)

for x ∈ L_{2}. The action of Π on Q × Π defined by the trivial action on Q and right-multiplication on Π descends to a free proper action of Π on M̃. This action corresponds to the action of deck transformations. Let ρ : Π ⟶ G be the homomorphism defined by:

ρ(X) = ξ
ρ(Y) = η

The embedding \begin{matrix}.... \\ Q \end{matrix} ⟶ Q together with the identifications of the sides of Q extends uniquely to a ρ-equivariant local homeomorphism M̃ ⟶ H^{2}. The holonomy around the puncture

ρ([X, Y]) = [ξ, η] = γ

is elliptic of rotation angle θ fixing p. The resulting hyperbolic structure extends to a singular hyperbolic structure on M̄ with a singularity of cone angle θ. Thus Theorem 3.4.1 reduces to Lemma 3.4.2. □

Figure 9. A convex embedded quadrilateral

Figure 10. When \overline{p_{3}p_{4}} meets \overline{p_{1}p_{2}}

.

Figure 11. When \overline{p_{2}p_{3}} meets \overline{p_{4}p_{1}}

Figure 12. A nonconvex embedded quadrilateral

The proof of Lemma 3.4.2 is based on the following two lemmas.

Lemma 3.4.3. The points p_{1}, p_{2}, p_{3}, p_{4} are not collinear.

Lemma 3.4.4. If the sequence of points p_{1}, p_{2}, p_{3}, p_{4} are not the vertices of an embedded quadrilateral, then either l_{1} and l_{3} intersect, or l_{2} and l_{4} intersect.

Proof of Lemma 3.4.3: Suppose that p_{1}, p_{2}, p_{3}, p_{4} are collinear. If more than one line contains p_{1}, p_{2}, p_{3}, p_{4}, then all the points coincide and ρ fixes this point, contradicting γ ≠ 𝕀. Thus a unique line l contains p_{1}, p_{2}, p_{3}, p_{4}. We claim that l is ρ-invariant.

If p_{3} = p_{4}, then η fixes p and since γ = ξηξ^{−1}η^{−1} fixes p, so must ξηξ^{−1} fix p. Since γ = [ξ, η] is nontrivial, both ξ and η are nontrivial. Thus ξ maps the unique fixed point of η to the unique fixed point of ξηξ^{−1}, and both ξ and η fix p, contradicting γ ≠ 𝕀. Thus p_{3} ≠ p_{4} and l is the unique line containing p_{3} and p_{4}.

ξ(p_{2}) = p_{3} and ξ(p_{1}) = p_{4} imply that p_{1} ≠ p_{2}. Thus l is the unique line containing p_{1} and p_{2}, and is ξ-invariant. Similarly l is η-invariant and thus ρ(π)-invariant, as claimed.

The commutator subgroup of the stabilizer of a line l consists of hyperbolic elements, contradicting ellipticity of γ. □

Proof of Lemma 3.4.4: If x, y ∈ H^{2} are distinct points, let \overleftrightarrow{xy} denote the line passing through x and y. If z ∉ \overleftrightarrow{xy}, let H_{x,y}(z) denote the component (a half-space) of H^{2} − \overleftrightarrow{xy} containing z, and Ĥ_{x,y}(z) denote the complementary component so that:

H^{2} = H_{x,y}(z) ⨿ \overleftrightarrow{xy} ⨿ Ĥ_{x,y}(z)

Suppose first that no three of the points p_{1}, p_{2}, p_{3}, p_{4} are collinear. There are four cases, depending on where p_{4} lies in relation to the decompositions determined by the lines \overleftrightarrow{p_{1}p_{2}} and \overleftrightarrow{p_{2}p_{3}}:

(1) p_{4} ∈ H_{p_{1}p_{2}}(p_{3}) ∩ H_{p_{2}p_{3}}(p_{1});

(2) p_{4} ∈ Ĥ_{p_{1}p_{2}}(p_{3}) ∩ H_{p_{2}p_{3}}(p_{1});

(3) p_{4} ∈ H_{p_{1}p_{2}}(p_{3}) ∩ Ĥ_{p_{2}p_{3}}(p_{1});

(4) p_{4} ∈ Ĥ_{p_{1}p_{2}}(p_{3}) ∩ Ĥ_{p_{2}p_{3}}(p_{1}). In the first case p_{1}, p_{2}, p_{3}, p_{4} are the vertices of a convex quadrilateral. (Compare Figure 9.) In the second case, \overline{p_{3}p_{4}} meets \overline{p_{1}p_{2}}. (Compare Figure 10.) In the third case, \overline{p_{2}p_{3}} meets \overline{p_{4}p_{1}}. (Compare Figure 11.) In the fourth case p_{1}, p_{2}, p_{3}, p_{4} are the vertices of an embedded (nonconvex) quadrilateral. (Compare Figure 12.)

Suppose next that three of the vertices are collinear. By Lemma 3.4.3, not all four vertices are collinear. By possibly conjugating ρ by ξ, η etc., we may assume that p_{1}, p_{2}, p_{3} are collinear. If p_{2} lies between p_{1} and p_{3}, then p_{1}, p_{2}, p_{3}, p_{4} are the vertices of an embedded quadrilateral (with a straight angle at p_{2}). Otherwise \overline{p_{2}p_{3}} meets \overline{p_{4}p_{1}} at p_{1}. This completes the proof of Lemma 3.4.4. □

Returning to the proof of Lemma 3.4.2, we show that l_{1} and l_{3} cannot intersect; an identical proof implies l_{2} and l_{4} cannot intersect.

Claim. Suppose that l_{1} and l_{3} intersect. Then l_{1} ∩ l_{3} is a point q and the triangles △(qp_{1}p_{4}) and △(qp_{3}p_{2}) are congruent.

Proof of Claim. If l_{1} ∩ l_{3} is not a point, then it is a segment, contradicting Lemma 3.4.3. Since ξ(l_{1}) = l_{3} and η(l_{2}) = l_{4}, the lengths of opposite sides are equal:

d(p_{1}, p_{2}) = ℓ(l_{1}) = ℓ(l_{3}) = d(p_{4}, p_{3}),
d(p_{2}, p_{3}) = ℓ(l_{2}) = ℓ(l_{4}) = d(p_{1}, p_{4})

Since the lengths of the corresponding sides are equal, the triangle △(p_{2}p_{1}p_{4}) is congruent to △(p_{4}p_{3}p_{2}). In particular

∠(p_{2}p_{1}p_{4}) = ∠(p_{4}p_{3}p_{2}).

Similarly, ∠(p_{3}p_{4}p_{1}) = ∠(p_{1}p_{2}p_{3}). Now

∠(qp_{1}p_{4}) = ∠(p_{2}p_{1}p_{4}) = ∠(p_{4}p_{3}p_{2}) = ∠(qp_{3}p_{2})

and similarly, ∠(qp_{4}p_{1}) = ∠(qp_{2}p_{3}). Since l_{4} = \overline{p_{1}p_{4}} is congruent to l_{2} = \overline{p_{3}p_{2}}, triangles △(qp_{1}p_{4}) and △(qp_{3}p_{2}) are congruent as claimed. □

Now let l be the line through q bisecting the angle ∠(p_{1}qp_{3}) such that reflection R in l interchanges △(qp_{1}p_{4}) and △(qp_{3}p_{2}). Since

R : \begin{cases}p_{1} & ⟼ p_{3} \\ p_{2} & ⟼ p_{4} \\ p_{3} & ⟼ p_{1} \\ p_{4} & ⟼ p_{2}, \end{cases}

ξ : \begin{cases}p_{1} & ⟼ p_{4} \\ p_{2} & ⟼ p_{3} \end{cases}

η : \begin{cases}p_{2} & ⟼ p_{1} \\ p_{3} & ⟼ p_{4}, \end{cases}

ξ ∘ R interchanges p_{3} and p_{4}, and R ∘ η interchanges p_{2} and p_{3}. Since an orientation-reversing isometry of H^{2} which interchanges two points must be reflection in a line, ξ ∘ R and R ∘ η have order two. Thus R conjugates ξ to ξ^{−1} and η to η^{−1}.

One of two possibilities must occur:

• At least one of ξ and η is elliptic or parabolic;

• Both ξ and η are hyperbolic, and their invariant axes are each orthogonal to l.

Neither possibility occurs, due to the following:

Lemma 3.4.5. Let ξ, η ∈ SL(2, ℝ). The following conditions are equivalent:

• tr[ξ, η] < 2;

• ξ, η are hyperbolic elements and their invariant axes cross.

Proof. Assuming tr[ξ, η] < 2, we first show that ξ and η must be hyperbolic. We first show that ξ is not elliptic. If ξ is elliptic (or ±𝕀), we may assume that ξ ∈ SO (2), that is, we represent ξ, η by matrices

ξ = [\begin{matrix}cos(θ) & − sin(θ) \\ sin(θ) & cos(θ) \end{matrix}] , η = [\begin{matrix}a & b \\ c & d \end{matrix}] ,

where ad − bc = 1, whence

tr[ξ, η] = 2 + sin^{2}(θ)(a^{2} + b^{2} + c^{2} + d^{2} − 2) ≥ 2.

Similarly if ξ is parabolic, (2.3.1) implies that tr[ξ, η] ≥ 2.

Thus ξ is hyperbolic. Since [η, ξ] = [ξ, η]^{−1}, an identical argument shows that η is hyperbolic. Denote their invariant axes by l_{ξ}, l_{η} respectively.

It remains to show that tr[ξ, η] < 2 if and only if l_{ξ} ∩ l_{η} ≠ ∅.

By conjugation, we may assume that the fixed points of ξ are ±1 and that the fixed points of η are r, ∞. Thus l_{ξ} ∩ l_{η} ≠ ∅ if and only if −1 < r < 1. Represent ξ, η by matrices

ξ = [\begin{matrix}cosh(θ) & sinh(θ) \\ sinh(θ) & cosh(θ) \end{matrix}] , η = [\begin{matrix}e^{ϕ} & −2r sinh(ϕ) \\ 0 & e^{−ϕ} \end{matrix}] ,

where θ, ϕ ≠ 0 and

tr[ξ, η] = 2 + 4(r^{2} − 1) sinh^{2}(θ) sinh^{2}(ϕ).

Then tr[ξ, η] < 2 if and only if −1 < r < 1 and tr[ξ, η] = 2 if and only if r = ±1, as desired. □

This completes the proof of Lemma 3.4.2 (and also Theorem 3.4.1). □ 3.5. Properness. That Γ acts properly on κ^{−1}(t) ∩ ℝ^{3} now follows easily. The construction in Theorem 3.4.1 gives a map

(3.5.1)  κ^{−1}(t) ∩ [2, ∞)^{3} ⟶ 𝔗(θ)

which is evidently Out(π)-equivariant. Every hyperbolic structure with conical singularities has an underlying singular conformal structure, where the singularities are again conical singularities, that is, they are defined by local coordinate charts to a model space, which in this case is a cone. However, there is an important difference. Conical singularities in conformal structures are removable, while conical singularities in Riemannian metrics are not.

Here is why conformal conical singularities are removable: Let D^{2} ⊂ ℂ be the unit disk and let

S_{θ} := {z ∈ D^{2} | 0 ≤ arg(z) ≤ θ}

be a sector of angle θ. Then

Π_{θ} : S_{θ} − {0} ⟶ D^{2} − {0}
z ⟼ z^{2π/θ}

is conformal. The model coordinate patch for a cone point of angle θ is the cone C_{θ} of angle θ, defined as identification space C_{θ} of S_{θ} by the equivalence relation defined by

z ⟷ e^{±iθ}z

for z ∈ ∂S_{θ}. That is, a cone point p has a coordinate patch neighborhood U and a coordinate chart ψ : U ⟶ C_{θ} in the atlas defining the singular geometric structure. The power map Π_{θ} defines a conformal isomorphism between the C_{θ} punctured at the cone point and a punctured disk (a “cone” of angle 2π). Replacing the coordinate chart ψ : U ⟶ ℂ at a cone point p of angle θ by the composition Π_{θ} ∘ ψ gives a coordinate atlas for a conformal structure which is nonsingular at p and isomorphic to the original structure on the complement of p.

The resulting map 𝔗(θ) ⟶ 𝔗_{M} is evidently Out(π)-equivariant. Since Out(π) acts properly on 𝔗_{M}, Out(π) acts properly on 𝔗(θ), and hence on κ^{−1}(t) ∩ ℝ^{3} as well.

With more work one can show that (3.5.1) is an isomorphism. For any hyperbolic structure on T^{2} with a cone point p of angle 0 < θ < 2π, the Arzelá-Ascoli theorem (as in Buser [5] §1.5) applies to represent ξ, η by geodesic loops based at p. Furthermore these geodesics intersect only at p. By developing this singular geometric structure one obtains a polygon Q as above.

4. Reducible characters (t = 2)

The level set κ^{−1}(2) consists of characters of reducible representations. Over ℂ such a representation is upper-triangular (2.3.2) with character defined by (2.3.3). By (2.3.4), every (x, y, z) ∈ κ^{−1}(2) lies in the image of the map

(4.0.2)  Φ : ℂ^{∗} × ℂ^{∗} ⟶ κ^{−1}(2)
(ξ, η) ⟼ [\begin{matrix}ξ + ξ^{−1} \\ η + η^{−1} \\ ξη + ξ^{−1}η^{−1} \end{matrix}] .

The set of ℝ-points κ^{−1}(2) ∩ ℝ^{3} is a singular algebraic hypersurface in ℝ^{3}, with singular set

S_{0} = \{[\begin{matrix}2 \\ 2 \\ 2 \end{matrix}] , [\begin{matrix}2 \\ −2 \\ −2 \end{matrix}] , [\begin{matrix}−2 \\ 2 \\ −2 \end{matrix}] , [\begin{matrix}−2 \\ −2 \\ 2 \end{matrix}]\} .

Characters in S_{0} correspond to unipotent representations twisted by central characters (as in the sense of (2.2.1)). A central character is a homomorphism taking values in the center {±𝕀} of G and a unipotent representation is a representation in ±U, where U is a unipotent subgroup of G. A reductive representation with character in S_{0} is itself a central character. The most general representation with character in S_{0} is one taking values in ±U, where U is a unipotent subgroup of G. The character (2, 2, 2) is the character of any unipotent representation, for example the trivial representation. The other three points are images of (2, 2, 2) by the three nontrivial elements of Σ.

The smooth stratum of κ^{−1}(2) is the complement

κ^{−1}(2) ∩ ℝ^{3} − S_{0}.

Denote its five components by C_{K} and C_{i}, where i = 0, 1, 2, 3. Here C_{0} denotes the component κ^{−1}(2) ∩ [2, ∞)^{3} and C_{i} = (σ_{i})_{∗}C_{0} where (σ_{i})_{∗} ∈ Σ is the sign-change automorphism fixing the i-th coordinate (§2.2.1).

The component C_{K} corresponds to reducible SU(2)-representations which are non-central, that is, their image does not lie in the center {±𝕀} of SU(2). The closure of C_{K} is the union of C_{K} with S_{0}. The map

U(1) × U(1) ⟶ C̄_{K}
(ξ, η) ⟼ (ξ + ξ^{−1}, η + η^{−1}, ξη + ξ^{−1}η^{−1})

is a double branched covering space, with deck transformation

(ξ, η) ⟼ (ξ^{−1}, η^{−1})

and four branch points

(ξ, η) = (±1, ±1)

which map to S_{0}.

Figure 13. Level sets of (x^{2} − 4)(y^{2} − 4)

Similarly, each of the other components identifies with the quotient (ℝ_{+})^{2}/{±𝕀} with the action of GL(2, ℤ). For example, C̄_{0} is the image of the double branched covering

ℝ_{+} × ℝ_{+} ⟶ C̄_{0}
(ξ, η) ⟼ (ξ + ξ^{−1}, η + η^{−1}, ξη + ξ^{−1}η^{−1}),

with deck transformation

(ξ, η) ⟼ (ξ^{−1}, η^{−1}).

For i = 1, 2, 3, the component C_{i} is the image of C_{0} under the signchange (σ_{i})_{∗}. Composing with the GL(2, ℤ)-equivariant diffeomorphisms

ℝ^{2} ⟶ ℝ_{+} × ℝ_{+}
(ξ̃, η̃) ⟼ (exp(ξ̃), exp(η̃))

and

(ℝ/ℤ)^{2} ⟶ U(1) × U(1)
(ξ̃, η̃) ⟼ (exp(2πiξ̃), exp(2πiη̃))

respectively, yields GL(2, ℤ)-equivariant double branched coverings

ℝ^{2} ⟶ C_{i}, and (ℝ/ℤ)^{2} ⟶ C_{K}

respectively. Furthermore these mappings pull back the invariant area form on κ^{−1}(2) ∩ ℝ^{3} to Lebesgue measure on ℝ^{2}.

Since SL(2, ℤ) is a lattice in SL(2, ℝ) and SL(2, ℝ) acts transitively on ℝ^{2} with noncompact isotropy group, Moore’s ergodicity theorem (Moore [27]; see also Feres [7], Zimmer [34] or Margulis [24]) implies that SL(2, ℤ) acts ergodically on ℝ^{2}. Thus GL(2, ℤ) acts ergodically on ℝ^{2}/{±𝕀} and (ℝ/ℤ)^{2}/{±𝕀}, and hence on each of the components

C_{0}, C_{1}, C_{2}, C_{3}, C_{K} ⊂ κ^{−1}(2) ∩ ℝ^{3}.

Since Σ permutes C_{0}, C_{1}, C_{2}, C_{3}, the Γ-action on their union

C_{0} ∪ C_{1} ∪ C_{2} ∪ C_{3} = κ^{−1}(2) ∩ ℝ^{3} − C_{K} ∪ S_{0}
≅ (ℝ^{∗} × ℝ^{∗})/{±𝕀}

is ergodic. (In the case of C_{K}, any hyperbolic element in GL(2, ℤ) acts ergodically on U(1) × U(1) and hence on C̄_{0}, a much stronger result.)

5. Three-holed spheres and ergodicity (t > 2)

Next we consider the level sets where t > 2. There is an important difference between the cases when t > 18 and 2 < t ≤ 18. When t ≤ 18, the Γ-action is ergodic, but when t > 18, wandering domains appear, arising from the Fricke spaces of a three-holed sphere P (“pairof-pants”). The three-holed sphere is the only other orientable surface homotopy-equivalent to a one-holed torus, and homotopy equivalences to hyperbolic manifolds homeomorphic to P define points in these level sets when t > 18, which we call discrete P -characters However, the Γaction on the complement of the discrete P -characters is ergodic.

5.1. The Fricke space of a three-holed sphere. When t > 18, the octant

Ω_{0} = (−∞, −2) × (−∞, −2) × (−∞, −2)

intersects κ^{−1}(t) in a wandering domain and the images of Ω_{0} ∩ κ^{−1}(t) are freely permuted by Γ. Let

Ω = Γ · Ω_{0}o.

Characters in Ω correspond to discrete embeddings ρ : π ⟶ SL(2, ℝ) where the complete hyperbolic surface H^{2}/ρ(π) is diffeomorphic to a three-holed sphere. We call such a discrete embedding a discrete P - embedding, and its character a discrete P -character.

The fundamental group π_{1}(P) is free of rank two. A pair of boundary components ∂_{1}, ∂_{2}, an orientation on P, and a choice of arcs α_{1}, α_{2} from the basepoint to ∂_{1}, ∂_{2} determines a pair of free generators of π_{1}(P):

X := (α_{1})^{−1} ⋆ ∂_{1} ⋆ α_{1}
Y := (α_{2})^{−1} ⋆ ∂_{2} ⋆ α_{2}.

A third generator Z := (XY)^{−1} corresponds to the third boundary component, obtaining a presentation of π_{1}(P) as

⟨X, Y, Z | XY Z = 1⟩.

Elements of Ω_{0} are discrete P -characters such that the generators X, Y, Z := (XY)^{−1} of π correspond to the boundary components of the quotient hyperbolic surface H^{2}/ρ(π).

Lemma 5.1.1. A representation ρ ∈ Hom(π, SL(2, ℂ)) has character [ρ] = (x, y, z) ∈ Γ · Ω_{0} if and only if ρ is a discrete P -embedding such that X, Y, Z correspond to the components of ∂H^{2}/ρ(π).

Proof. The condition that [ρ] ∈ Ω_{0} is equivalent to x, y, z < −2, which implies that the generators ρ(X), ρ(Y) and ρ(XY) are hyperbolic and their invariant axes l_{ρ(X)}, l_{ρ(Y)}, l_{ρ(XY)} are pairwise ultraparallel. Denoting the common perpendicular to two ultraparallel lines l, l′ by ⊥ (l, l′), the six lines

l_{ρ(X)}, ⊥(l_{ρ(X)}, l_{ρ(Y)}), l_{ρ(Y)}, ⊥(l_{ρ(Y)}, l_{ρ(XY)}), l_{ρ(XY)}, ⊥(l_{ρ(XY)}, l_{ρ(X)})

bound a right-angled hexagon H. The union of H with its reflected image in ⊥ (l_{ρ(X)}, l_{ρ(Y)}) is a fundamental domain for ρ(π) acting on H^{2}. (Goldman [11], Gilman-Maskit [10], I-7, p. 15). (Figure 14 depicts the identifications corresponding to the generators ρ(X), ρ(Y).) The quotient is necessarily homeomorphic to a three-holed sphere P and the holonomy around components of ∂P are the three generators ρ(X), ρ(Y), ρ(XY). □

Recall (§2.2.2) that Γ decomposes as the semidirect product

Γ ≅ Γ_{(2)} ⋉ 𝔖_{3}.

The mapping class group of a three-holed sphere P is isomorphic to ℤ/2 × 𝔖_{3}. The 𝔖_{3}-factor corresponds to the group of permutations of π_{0}(∂P). The ℤ/2-factor is generated by the elliptic involution which Figure 14. Fundamental hexagons for a hyperbolic three-holed sphere

acts by an orientation-reversing homeomorphism of P, whose fixedpoint set is the union of three disjoint arcs joining the boundary components. (See §2.2.3 for the corresponding automorphism of π.) Accordingly, 𝔖_{3} preserves the Teichmüller space of P. The following result shows that these are the only automorphisms of the character variety which preserve the discrete P -characters.

Proposition 5.1.2. Ω equals the disjoint union ∐_{γ∈Γ_{(2)}} γΩ_{0}.

Proof. We show that if for some γ ∈ Γ_{(2)}, the intersection Ω_{0} ∩ γΩ_{0} is nonempty, then γ = 1. Suppose that [ρ] ∈ Ω_{0} ∩ γΩ_{0}. By Lemma 5.1.1, both ρ and ρ ∘ γ are discrete P -embeddings such that X, Y, (XY)^{−1} and γ(X), γ(Y), γ((XY)^{−1}) correspond to ∂H^{2}/ρ(π).

The automorphism γ of the character space ℂ^{3} corresponds to an automorphism γ̃ of π such that

[ρ ∘ γ̃] = σ_{∗} ∘ γ([ρ])

for a sign-change automorphism σ_{∗} ∈ Σ. Thus ρ ∘ γ̃ is also a discrete P -embedding, with quotient bounded by curves corresponding to γ̃(X), γ̃(Y) and γ̃(XY). Then γ̃(X) (respectively γ̃(Y),γ̃(XY)) is conjugate to X^{±1} (respectively Y^{±1}, (XY)^{±1}). Such an automorphism is induced by a diffeomorphism of the three-holed sphere P. Since the elliptic involution generates the mapping class group of P, γ̃ must be an inner automorphism possibly composed with the elliptic involution of π, and hence must act trivially on characters. Thus γ = 1 as desired. □

For t ≤ 18, the domain Ω does not meet κ^{−1}(t). (The closure Ω̄ intersects κ^{−1}(18) in the Γ-orbit of the character (−2, −2, −2) of the holonomy representation of a complete finite-area hyperbolic structure on P.) For t > 18, observe that κ^{−1}(t) − Ω contains the open subset

((−2, 2) × ℝ × ℝ) ∩ κ^{−1}(t).

Thus κ^{−1}(t) − Ω has positive measure in κ^{−1}(t).

Proposition 5.1.3. For any t > 2, the action of Γ on κ^{−1}(t) − Ω is ergodic.

The proof uses an iterative procedure (Theorem 5.2.1) due to Kern-Isberner and Rosenberger [18], although their proof contains a gap near the end. See also Gilman and Maskit [10]. Theorem 5.2.1 is proved at the end of the paper.

5.2. The equivalence relation defined by Γ. Write u ∼ v if there exists γ ∈ Γ such that γu = v. Since κ is Γ-invariant, u ∼ v implies that κ(u) = κ(v).

Theorem 5.2.1. Suppose that u ∈ ℝ^{3} satisfies κ(u) > 2. Then there exists (x̃, ỹ, z̃) = ũ ∼ u such that either

• ũ ∈ (−∞, −2]^{3}, in which case ũ is the character of a Fuchsian representation whose quotient is a hyperbolic structure on a three-holed sphere P, with boundary mapping to either cusps or closed geodesics;

• x̃ ∈ [−2, 2], in which case ũ is the character of a representation mapping X to a non-hyperbolic element.

Recall that a measurable equivalence relation is ergodic if the only invariant measurable sets which are unions of equivalence classes are either null or conull. Equivalently an equivalence relation is ergodic if and only if every function constant on equivalence classes is constant almost everywhere. A group action defines an equivalence relation. However, equivalence relations are more flexible since every subset of a space with an equivalence relation inherits an equivalence relation (whether it is invariant or not). Suppose that S is a measurable subset of a measure space X. If every point in X is equivalent to a point of a measurable subset of S, then ergodicity of the equivalence relation on X is equivalent to ergodicity of S with respect to the measure class induced from X.

Proof of Proposition 5.1.3 assuming Theorem 5.2.1. For i = 1, 2, let 𝔈_{t}^{(i)} denote the subset of κ^{−1}(t) − Ω where at least i of the coordinates lie in [−2, 2]. Suppose that t > 2 and u ∈ (κ^{−1}(t) − Ω). Then Theorem 5.2.1 implies that Γu ∩ 𝔈_{t}^{(1)} ≠ ∅. Thus ergodicity of the Γ-action on κ^{−1}(t) − Ω is equivalent to the ergodicity of the induced equivalence relation on 𝔈_{t}^{(1)}.

We now reduce to the equivalence relation on 𝔈_{t}^{(2)}. By applying a permutation we may assume that −2 < x < 2.

The level set

E(x_{0}) := κ^{−1}(t) ∩ x^{−1}(x_{0})

is defined by

\frac{2 − x_{0}}{4} (y + z)^{2} + \frac{2 + x_{0}}{4} (y − z)^{2} = t − 2 + x_{0}^{2}

and is an ellipse since −2 < x_{0} < 2. Furthermore the symplectic measure on κ^{−1}(t) disintegrates under the map x : κ^{−1}(t) ⟶ ℝ to τ_{X}-invariant Lebesgue measure on E(x_{0}). In particular the Dehn twist τ_{X} (see §2.2.3) acts by the linear map

[\begin{matrix}y \\ z \end{matrix}] ⟶ [\begin{matrix}x & −1 \\ 1 & 0 \end{matrix}] [\begin{matrix}y \\ z \end{matrix}] ,

which is linearly conjugate to a rotation of the circle through angle cos^{−1}(x/2). Thus for almost every x_{0} ∈ (−2, 2) the Dehn twist τ_{X} acts on E(x_{0}) by a rotation of infinite order, and the action is ergodic. Furthermore by applying powers of τ_{X}, we may assume that −2 < y < 2 as well. Thus ergodicity of the Γ-action on κ^{−1}(t) − Ω is equivalent to the ergodicity of the induced equivalence relation on 𝔈_{t}^{(2)}.

Since Γ acts by polynomial transformations over ℤ, those points of κ^{−1}(t) which are equivalent to a point with cos^{−1}(x/2) is rational comprise a set of measure zero. We henceforth restrict to the complement of this set.

The quadratic reflection (see §2.2.3)

Q_{z} : [\begin{matrix}x \\ y \\ z \end{matrix}] ⟼ [\begin{matrix}x \\ y \\ xy − z \end{matrix}]

is the deck transformation for the double covering of κ^{−1}(t) given by projection Π_{(x,y)} to the (x, y)-plane. The image

Π_{(x,y)} : κ^{−1}(t) ⊂ ℝ^{3} ⟶ ℝ^{2}

is the region

R_{t} := {(x, y) ∈ ℝ^{2} | (x^{2} − 4)(y^{2} − 4) + t − 2 ≥ 0}

and Π_{(x,y)} : κ^{−1}(t) ⟶ R_{t} is the quotient map for the action of Q_{z}. Thus ergodicity on 𝔈_{t}^{(2)} reduces to ergodicity of the induced equivalence relation on

Π_{(x,y)}(𝔈_{t}^{(2)}) = [−2, 2] × [−2, 2].

Ergodicity now follows as in §5.2 of [13]. Suppose that f : κ^{−1}(t) − Ω ⟶ ℝ is a Γ-invariant measurable function. The ergodic decomposition for the equivalence relation induced by the cyclic group ⟨τ_{X}⟩ is the coordinate function

x : [−2, 2] × [−2, 2] ⟶ [−2, 2],

and by ergodicity of ⟨τ_{X}⟩ on the level sets of x, there is a measurable function g : [−2, 2] ⟶ ℝ such that f factors as f = g ∘ x almost everywhere. Applying the cyclic group ⟨τ_{Y} ⟩ to [−2, 2], the function g is constant almost everywhere. Hence f is constant almost everywhere, proving ergodicity. □

5.3. The trace-reduction algorithm. The proof of Theorem 5.2.1 is based on the following:

Lemma 5.3.1. Let 2 < x ≤ y ≤ z. Suppose that κ(x, y, z) > 2. Let z′ = xy − z. Then z − z′ > 2\sqrt{κ(x, y, z) − 2}.

The following expression for κ(x, y, z) will be useful:

(5.3.1)  κ(x, y, z) − 2 = \frac{1}{4} \{(2z − xy)^{2} − (x^{2} − 4)(y^{2} − 4)\}

= \frac{1}{4} \{(z − z′)^{2} − (x^{2} − 4)(y^{2} − 4)\} .

For fixed x, y > 2, write κ_{x,y} : ℝ → ℝ for the quadratic function

κ_{x,y} : z ⟼ κ(x, y, z) − 2 = z^{2} − xyz + (x^{2} + y^{2} − 4).

Then

κ_{x,y}^{−1}((−∞, 0])) = [ζ_{−}(x, y), ζ_{+}(x, y)]

where

ζ_{±}(x, y) = \frac{1}{2} (xy ± \sqrt{(x^{2} − 4)(y^{2} − 4)}) .

Furthermore reflection

z ⟼ z′ = xy − z

interchanges the two intervals

J_{−} = ( − ∞, ζ_{−}(x, y))
J_{+} = (ζ_{+}(x, y), ∞)

comprising κ_{x,y}^{−1}((0, ∞)).

Lemma 5.3.2. Suppose that 2 < x ≤ y. Then ζ_{−}(x, y) < y < ζ_{+}(x, y)

Proof. The conclusion is equivalent to κ(x, y, y) < 2, which is what we prove. First observe that y > 2 implies y − 1/2 > 3/2 so that

y − y^{2} = − (y − \frac{1}{2})^{2} + \frac{1}{4}

< − (\frac{3}{2})^{2} + \frac{1}{4} = −2

and x < y implies that x + 2 − y^{2} < y + 2 − y^{2} < 0. Therefore

κ(x, y, y) − 2 = x^{2} + 2y^{2} − xy^{2} − 4
= (x^{2} − 4) + (2y^{2} − xy^{2})
= (x − 2)(x + 2 − y^{2}) < 0.
□

Conclusion of Proof of Lemma 5.3.1. By Lemma 5.3.2, the quadratic function κ_{x,y} is negative at y. By hypothesis κ_{x,y} is positive at z ≥ y. Therefore

z > ζ_{+}(x, y) > y.

Thus z ∈ J_{+}. Reflection z ⟼ z′ interchanges the intervals J_{+} and J_{−}, so z′ ∈ J_{−}, that is z′ < ζ_{−}(x, y) < y (Lemma 5.3.2). By (5.3.1),

(z − z′)^{2} = 4(κ(x, y, z) − 2) + (x^{2} − 4)(y^{2} − 4)
> 4(κ(x, y, z) − 2)

whence (because z ≥ y > z′)

z − z′ > 2\sqrt{κ(x, y, z) − 2},

completing the proof of Lemma 5.3.1. □

Conclusion of Proof of Theorem 5.2.1. Fix

u = (x, y, z) ∈ ℝ^{3}

with κ(u) > 2. We seek ũ ∼ u such that one of the following possibilities occurs:

• one of the coordinates x̃, ỹ, z̃ lies in the interval [−2, 2];

• x̃, ỹ, z̃ < −2. It therefore suffices to find ũ ∼ u which lies in (−∞, 2]^{3}. Suppose that u does not satisfy this. The linear automorphisms in Γ are arbitrary permutations of the coordinates and sign-change automorphisms which allow changing the signs of two coordinates. By applying linear automorphisms, we can assume that 2 < x ≤ y ≤ z.

Let μ := 2\sqrt{κ(u) − 2} > 0. By Lemma 5.3.1 the quadratic reflection Q_{z} ∈ Γ given by

u = [\begin{matrix}x \\ y \\ z \end{matrix}] ⟼ ũ = [\begin{matrix}x \\ y \\ z′ \end{matrix}]

reduces z by more than μ. If z′ ≤ 2 then

ũ ∼ (−x, −y, z′) ∈ (−∞, 2]^{3},

completing the proof. Otherwise, all three coordinates of ũ are greater than 2 so we repeat the process. Since each repetition decreases x+y+z by more than μ, the procedure ends after at most (x + y + z − 6)/μ steps. □

References

1. Abikoff, W., “The Real Analytic Theory of Teichmüller Space,” Lecture Notes in Mathematics 820 (1980) Springer-Verlag, Berlin and New York.

2. Bers, L. and Gardiner, F., Fricke spaces, Adv. Math. 62 (1986), 249–284.

3. Bowditch, B. H., Markoff triples and quasi-Fuchsian groups, Proc. London Math. Soc. (3) 77 (1998), no. 3, 697–736.

4. Brumfiel, G. and Hilden, H., “SL(2)-representations of finitely presented groups,” Contemp. Math. 187 Amer. Math. Soc., Providence, RI (1995).

5. Buser, P., “Geometry and Spectra of Compact Riemann Surfaces,” Progress in Mathematics 106, Birkhäuser Boston (1992).

6. Culler, M. and Shalen, P., Varieties of group representations and splittings of 3-manifolds, Ann. Math. 117 (1983),109–146.

7. Feres, R., “Dynamical Systems and Semisimple Groups,” Cambridge Tracts in Pure Mathematics 126, Cambridge University Press (1998).

8. Fricke, R., Über die Theorie der automorphen Modulgrupper, Nachr. Akad. Wiss. Göttingen (1896), 91–101.

9. and Klein, F., Vorlesungen der Automorphen Funktionen, Teubner, Leipzig, Vol. I (1897), Vol. II (1912).

10. Gilman, J. and Maskit, B., An algorithm for 2-generator Fuchsian groups, Michigan Math. J. 38 (1) (1991), 13–32

11. Goldman, W., “Discontinuous groups and the Euler class,” Doctoral dissertation, University of California, Berkeley (1980).

12. , Topological components of spaces of representations, Inv. Math. 93 (3) (1988), 557–607.

13. , Ergodic theory on moduli spaces, Ann. Math. 146 (1997), 475–507.

14. , An exposition of results of Fricke, (in preparation).

15. and Neumann, W., Homological action of the automorphism group of some cubic moduli spaces, (in preparation).

16. Harvey, W. J., Spaces of Discrete Groups, in “Discrete Groups and Automorphic Functions,” Academic Press (1977), 295–347.

17. Horowitz, R.D., Induced automorphisms on Fricke characters of Free groups, Trans. A.M.S. 208 (1975), 41–50.

18. Kern-Isberner, G. and Rosenberger, G., Über Diskretheitsbedingungen und die Diophantische Gleichung ax^{2} + by^{2} + cz^{2} = dxyz, Archiv. Math. (Basel) 34 (1980), no. 6, 481–493.

19. Imayoshi,Y. and Taniguchi, M., “An Introduction to Teichmüller spaces,” Springer-Verlag Tokyo Berlin Heidelberg New York (1992).

20. Lubotzky, A. and Magid, A., “Varieties of representations of finitely generated groups,” Mem. A. M. S. 336 (58) (1985).

21. Lyndon, R. and Schupp, R., “Combinatorial Group Theory,” Springer-Verlag (1977), Springer-Verlag Berlin Heidelberg New York.

22. Magnus, W., Rings of Fricke characters and automorphism groups of free groups, Math. Zeit. 170 (1980), 91–103.

23. , Karrass,A. and Solitar, D., “Combinatorial Group Theory: Presentations of groups in terms of generators and relations,” (1976) Dover Publications, New York.

24. Marguils, G., “Discrete Subgroups of Semisimple Groups,” Ergebniße der Math. 17, Springer-Verlag Berlin-Heidelberg-New York (1991).

25. McOwen, R., Prescribed curvature and singularities of conformal metrics on Riemann surfaces, J. Math. Anal. Appl. 177 (1993), no. 1, 287–298.

26. Morgan, J. W. and Shalen, P. B., Valuations, trees and degenerations of hyperbolic structures I, Ann. Math. 120 (1984), 401–476.

27. Moore, C.C. Ergodicity of flows on homogeneous spaces, Amer. J. Math. 88 (1966), 154–178.

28. Nag, S., “The Complex Analytic Theory of Teichmüller Spaces,” Can. Math. Soc. Ser. of Monographs and Advanced Texts, (1988) John Wiley & Sons, New York, Chichester, Brisbane, Toronto, Singapore.

29. Nielsen, J., Die Isomorphismen der allgemeinen unendlichen Gruppe mit zwei Erzeugenden, Math. Ann. 71 (1918), 385–397.

30. , Untersuchungen zur Topologie der geschlossenen zweiseitigen Flächen I, Acta Math. 50 (1927), 189–358.

31. Stantchev, G., “Dynamics of the modular group acting on GL(2, ℝ)-characters of a once-punctured torus,” Doctoral Dissertation, University of Maryland 32. Stillwell, J., “The Dehn-Nielsen theorem,” Appendix to Papers on Group The-

ory and Topology, by Max Dehn, Springer Verlag (1987) Berlin Heidelberg New York.

33. Troyanov, M., Prescribing curvature on compact surfaces with conical singularities, Trans. Amer. Math. Soc. 324 (2) (1991), 793–821.

34. Zimmer, R., “Ergodic theory and semisimple groups,”(1984) Monographs in Mathematics 81 Birkhäuser Boston Basel Stuttgart.

Mathematics Department, University of Maryland, College Park, MD 20742 USA

  E-mail address: wmg@math.umd.edu