Carnot metrics, dynamics and local rigidity

Abstract

This paper develops new techniques for studying smooth dynamical systems in the presence of a Carnot–Carathéodory metric. Principally, we employ the theory of Margulis and Mostow, Métivier, Mitchell, and Pansu on tangent cones to establish resonances between Lyapunov exponents. We apply these results in three different settings. First, we explore rigidity properties of smooth dominated splittings for Anosov diffeomorphisms and flows via associated smooth Carnot–Carathéodory metrics. Second, we obtain local rigidity properties of higher hyperbolic rank metrics in a neighborhood of a locally symmetric one. For the latter application we also prove structural stability of the Brin–Pesin asymptotic holonomy group for frame flows. Finally, we obtain local rigidity properties for uniform lattice actions on the ideal boundary of quaternionic and octonionic symmetric spaces.

1 Introduction

Totally non-integrable k-plane distributions E on a manifold N have been of considerable interest in partial differential equations, geometry and control theory. In particular, they play a central role in Hörmander’s work on hypoelliptic partial differential operators (e.g. [Hör67]). Such operators arise naturally from nilpotent groups. Fundamental work of Folland, Goodman, Kohn, and Rothchild and Stein developed a converse.

Later on, Gromov [Gro81] introduced tangent cones for general metric spaces and found criteria for their existence. Namely, for a metric space $(X,d)$ a tangent cone at $p\in X$ is a convergent limit in the pointed Gromov–Hausdorff topology of the sequence of pointed metric spaces $(X,p,\lambda d)$ as $\lambda \to \infty $ . Note that tangent cones do not always exist, and when they do exist, they are not always unique. Spun from a key result of Métivier [ Mét76 ], Gromov, Mitchell, Margulis and Mostow, and Pansu, among others, have investigated the existence, uniqueness and structure of the pointed tangent cones for Carnot–Carathéodory metrics [Gro81, Gro96, MM00, MM95, Mit85, Pan89].

In the present paper we explore these tangent cones with the goal of establishing new techniques for the study of Lyapunov exponents of dynamical systems respecting sub-Riemannian metrics. We suspect that such systems will typically possess algebraic features. For example, we use these techniques to establish criteria for the Lyapunov spectra to be arithmetic progressions. This fits into the scope of the recent dueling programs between flexibility and rigidity of Lyapunov exponents. Indeed, pointwise arithmeticity of the Lyapunov spectrum sometimes leads to complete rigidity of the dynamics [But19, But18]. Further afield, constancy or extremality of the Lyapunov spectrum in certain cases yields local rigidity (cf. [Con03, DeW19, DSLVY20, GKS11, GKS20, SY19]). These contrast with results about flexibility of the Lyapunov spectrum for tori and surfaces, (see [BE20, BKH21, CS21, EK19]).

The arithmeticity of Lyapunov spectra has ramifications for both Riemannian geometry and group actions. Indeed, we will discuss rigidity of hyperbolic rank structures in §1.3 and local rigidity of projective actions in §1.4.

1.1 Sub-Riemannian geometry

We recall that a smooth manifold N is called a sub-Riemannian manifold if

1. (1) N is equipped with a smooth distribution E, called a horizontal distribution, which satisfies Hörmander’s condition (i.e., vector fields tangent to E and their iterated brackets generate $TN$ ) and

2. (2) E is endowed with a smooth Riemannian metric $\langle \cdot ,\cdot \rangle _x$ .

The Carnot–Carathéodory metric $d_C$ on N between a pair of points x and y is defined as the infimum of length of curves tangent to E from x to y. By a theorem of Chow [Cho39], the Carnot–Carathéodory metric $d_C$ is finite on connected components of N.

For a horizontal distribution E, at each point $x\in N$ and $i\geq 1$ we define $E^i_x$ to be the subspace of $T_xN$ spanned by all i-fold commutators of vector fields tangent to E. We call a horizontal distribution E generic (of order r) at a point $p\in N$ if on a neighborhood U of p the subspaces $E^i_x$ fit together to form a strictly increasing sequence of (smooth) subbundles

$$ \begin{align*}E=E^0\subset E^1 \subset \cdots \subset E^r=TU.\end{align*} $$

In this setting, Mitchell [Mit85] showed that the tangent cones at generic points of N are graded nilpotent Lie groups equipped with left-invariant Carnot–Carathéodory metrics which are unique by Margulis and Mostow [MM00]. Following Pansu [Pan89], Margulis and Mostow also developed the notion of a Pansu derivative of a quasi-conformal map on a sub-Riemannian manifold. We will be applying this construction to maps preserving a distribution.

We develop these ideas to understand the dynamics of differentiable maps preserving suitable distributions. Combining the main theorem of Mitchell [Mit85] together with Proposition 3.4 and analyzing (cf. Proposition 3.8) the proof of the main theorem of Margulis and Mostow [MM95] provides the catalyst for our investigations.

Theorem 1.1. (Tangent cone structure theorem)

Let $f:N\to N$ be a local $C^\infty $ diffeomorphism of a sub-Riemannian manifold N preserving the horizontal distribution E. Then the set $\Omega :=\{x\in N: E \text { is generic at } x\}$ is open dense and f-invariant. Moreover, we have the following statements.

1. (1) For every $x\in \Omega $ , the tangent cone $TC_xN$ of N at x exists, is a graded nilpotent Lie group with a left invariant Carnot–Carathéodory metric and the tangent cones vary continuously on $\Omega $ with respect to the Gromov–Hausdorff topology.

2. (2) The map f induces a graded Lie group isomorphism $(f_*)_x:TC_xN\to TC_{f(x)}N$ between the tangent cones $TC_xN$ of N at $x\in \Omega $ and $TC_{f(x)}N$ of N at $f(x)$ . We call $(f_*)_x$ the Carnot derivative of f at x.

With substantially more work, we later prove a foliated version of this theorem (Theorem 4.2). Applying this to an unstable foliation will provide applications in the dynamical setting.

Next we consider the relation between the Carnot derivative and the ordinary one. In the following three theorems we employ a detailed investigation of the proof of the Margulis–Mostow theorem [MM95].

Theorem 1.2. (Subadditivity of the spectrum at periodic points)

Let $f:N\to N$ be a local $C^\infty $ diffeomorphism of a sub-Riemannian manifold N preserving the horizontal distribution E. Assume that E is generic at a point $p\in N$ where $f(p)=p$ . Suppose the tangent cone $TC_pN$ is $(r+1)$ -step and set $n_{-1}=0$ and $n_i=\dim E^{i}(p)$ for $i\in \{ 0,\ldots ,r \}$ . If has Lyapunov exponents $(\log \lambda _1,\log \lambda _2,\ldots ,\log \lambda _{n_0},\log \lambda _{{n_0}+1},\ldots ,\log \lambda _{n_1}, \ldots ,\log \lambda _{n_{i}})$ listed with multiplicity and with $\log \lambda _{n_{j-1}+1},\ldots ,\log \lambda _{n_{j}}$ in non-decreasing order for each $j\in \{ 0,\ldots ,i \}$ , then for each $i\in \{ 0,\ldots , r \}$ and $j\in \{ n_{i-1}+1,\ldots ,n_{i} \}$ we have $(i+1) \log \lambda _1\leq \log \lambda _{j}\leq (i+1)\log \lambda _{n_0} $ .

Note that at a fixed point, the Lyapunov exponents simply amount to being the logs of the magnitude of the eigenvalues of $D_pf$ . In the special case when the tangent cone is the Heisenberg group with its standard contact structure and Carnot–Carathéodory metric, we have a stronger statement.

Theorem 1.3. (Additivity of the spectrum for Heisenberg cones at periodic points)

Let $f:N\to N$ be a local $C^\infty $ diffeomorphism of a sub-Riemannian manifold N preserving the horizontal distribution E. Assume that E is generic at a point $p\in N$ where $f(p)=p$ . Suppose the tangent cone $TC_pN$ is isomorphic to the Heisenberg group $H^{2n+1}$ . If $\log \lambda _1, \log \lambda _2, \ldots , \log \lambda _{2n}$ are the Lyapunov exponents of listed with multiplicity and $\log \lambda _{2n+1}$ is the remaining Lyapunov exponent of $D_pf$ , then $\log \lambda _1+\log \lambda _2+\cdots +\log \lambda _{2n}=n \log \lambda _{2n+1}$ .

For our next result, given an $(r+1)$ -step graded nilpotent Lie group $\mathcal {N}$ , we denote its graded Lie algebra by $\mathfrak {n}=\bigoplus _{i=0}^{r} \mathfrak {n}^i$ where $\mathfrak {n}^0,\mathfrak {n}^1=[\mathfrak {n}^0,\mathfrak {n}^0],\ldots ,\mathfrak {n}^{r}=[\mathfrak {n}^{r},\mathfrak {n}^0]$ . Suppose $f:N\to N$ is a diffeomorphism of a sub-Riemannian manifold N preserving its distribution E. If the Carnot derivative of f at a fixed point where E is generic is a homothety, then the Lyapunov spectrum is arithmetic.

Theorem 1.4. (Arithmeticity of the spectrum at periodic points)

Let $f:N\to N$ be a local $C^\infty $ diffeomorphism of a sub-Riemannian manifold N preserving the horizontal distribution E. Assume that E is generic at a point $p\in N$ where $f(p)=p$ , and that the graded automorphism $f_*:TC_pN\to TC_pN$ induced from f is a homothety. Then there exists a $\lambda>1$ such that has Lyapunov exponents $\log \lambda ,2\log \lambda ,\ldots ,(i+1) \log \lambda $ with corresponding multiplicities $\dim \mathfrak {n}^0,\ldots ,\dim \mathfrak {n}^{i}$ for $i=0,\ldots ,r$ .

We call a graded nilpotent Lie group asymmetric (cf. Definition 2.6 and Lemma 2.7) if its group of graded automorphisms consists of homotheties with respect to some Carnot–Carathéodory metric induced by an inner product on $\mathfrak {n}^0$ . The asymmetric groups are open in the space of isomorphism classes of nilpotent groups with a given grading, and sometimes non-empty (cf. Examples 2.8 and Corollary 2.10). Since Theorem 1.4 holds for any choice of sub-Riemannian metric on N, induced from an inner product on E, we have the following corollary.

Corollary 1.5. In the above theorem, when $TC_pN$ is an asymmetric nilpotent group, the conclusion holds without the assumption that the induced graded automorphism $f_*$ is a homothety.

We will apply the natural extension of this theorem to the case of a $C^1$ diffeomorphism $f:X\to X$ (or flow $\varphi _t: X\to X$ ) leaving invariant a foliation $\mathcal {F}$ with $C^\infty $ leaves tangent to a continuous distribution D. If $E\subseteq D$ is a $df$ -invariant continuous subdistribution, we define the set

$$ \begin{align*} \Omega&=\{ v\in X: E \text{ is } C^\infty \text{ in a neighborhood of } v \text{ in } \mathcal{F}(v)\\ &\quad\ \ \text{and is horizontal and generic there} \}. \end{align*} $$

Suppose that the distribution E is uniformly $C^k$ along $\mathcal {F}$ (cf. Definition 4.4) for a sufficiently large k and that $\mathcal {F}$ is transversally Hölder continuous. Also assume that f (or the flow $\varphi _t$ ) is $C^\infty $ along leaves, topologically transitive, satisfies the stable closing property (cf. Definition 4.7), and that is transversally Hölder continuous.

In this setting, the foliated tangent cone structure theorem (Theorem 4.2; cf. Corollary 4.6) provides a common isomorphism class, associated to the distribution D, of graded nilpotent Lie group structures for all tangent cones $TC_v\mathcal {F}(v)$ with $v\in \Omega $ and whose metrics vary continuously on $\Omega $ . This leads to our main rigidity result for Lyapunov exponents along leaves of the foliation.

Theorem 1.6. (Arithmeticity of the Lyapunov spectrum)

Suppose that the graded nilpotent group associated to D on $\Omega $ is asymmetric and $(r+1)$ -step. Let $\mu $ be any finite f-invariant (respectively, $\varphi _t$ -invariant) ergodic measure whose support $\operatorname {Supp}(\mu )$ satisfies $\operatorname {Supp}(\mu )\cap \Omega \neq \emptyset $ . Then the Lyapunov exponents of f (respectively, $\phi _1$ ) along $\mathcal {F}$ are $\log \lambda , 2\log \lambda , \ldots , (r+1)\log \lambda $ with multiplicities $\dim E,\dim E^{1}-\dim E,\ldots ,\dim E^{r}-\dim E^{r-1}$ for some $\lambda =\lambda _\mu>1$ .

We now consider some applications of the above results.

1.2 Smooth slow distributions

One family of examples where asymmetric nilpotent groups naturally arise are the quaternionic hyperbolic and octonionic hyperbolic symmetric spaces. Here the tangent cones of the stable and unstable foliations of a geodesic flow on compact quotients of these spaces are naturally isomorphic to certain asymmetric nilpotent groups of Heisenberg type (see Examples 2.8 below) corresponding to the nilradical of the full isometry group.

In the case when a flow nearby one of these symmetric flows still has smooth slow distribution we show it is orbit equivalent to the symmetric one.

Theorem 1.7. Let $\phi _t^0$ be the geodesic flow on a locally quaternionic hyperbolic or octonionic hyperbolic closed manifold M. Then if $\phi _t$ is any $C^\infty $ flow $C^1$ -close to $\phi _t^0$ for which $E^u_{\mathrm {slow},\phi _t}$ remains $C^\infty $ along unstable leaves and is sufficiently uniformly $C^1$ -close (in the sense of Definition 5.2) to that of $\phi _t^0$ , then $\phi _t$ is $C^\infty $ orbit equivalent to $\phi _t^0$ .

We suspect that strong rigidity properties hold whenever slow (un)stable distributions are smooth along leaves of the (un)stable foliation.

Questions 1.8. Can we characterize Anosov or partially hyperbolic diffeomorphisms or flows with smooth $E^u_{\mathrm {slow}}$ distributions along the unstable foliation? When are these smoothly orbit equivalent to algebraic ones? Is it sufficient that $E^u_{\mathrm {slow}}$ be smooth along unstable leaves and completely non-integrable there?

Note that there are simple non-algebraic examples coming from smooth time changes and suspensions of products (see §5.1). We also remark that there are numerous algebraic Anosov examples (e.g. on tori or products) where the slow distribution is integrable.

We obtain stronger rigidity properties if we restrict to geodesic flows.

Theorem 1.9. Let $g_0$ be a locally quaternionic hyperbolic or octonionic hyperbolic metric on a smooth closed manifold M. Then $g_0$ is locally rigid within the family of $C^2$ -close $C^\infty $ metrics whose $E^u_{\mathrm {slow}}$ remains $C^\infty $ along unstable leaves and is sufficiently uniformly $C^1$ -close (in the sense of Definition 5.2) to that of $g_0$ .

Another class we can handle are Anosov diffeomorphisms on nilmanifolds arising from asymmetric nilpotent groups. Let M be a closed manifold and let $f_0$ be a transitive $C^\infty $ Anosov diffeomorphism such that tangent cones of unstable leaves exist everywhere and are isomorphic to a fixed asymmetric $(r+1)$ -step Carnot nilpotent Lie group N. Let $E^u_{\mathrm {slow},f_0}$ be the horizontal distribution along unstable leaves that gives rise to the (graded) structure of N. (As it turns out, $E^u_{\mathrm {slow},f_0}$ is the slow distribution of smallest dimension which is still horizontal.) We obtain the following local spectral rigidity theorem.

Theorem 1.10. There is a $C^1$ open neighborhood U of $f_0$ in $\mathrm {Diff}{}^\infty (M)$ such that if $f \in U$ admits a smooth splitting $E^u_f=E^u_{\mathrm {fast},f}\oplus E^u_{\mathrm {slow},f}$ along unstable leaves with $\dim (E^u_{\mathrm {slow},f})=\dim (E^u_{\mathrm {slow},f_0})$ , and $E^u_{\mathrm {slow},f}$ is sufficiently uniformly $C^{r}$ -close along unstable leaves to $E^u_{\mathrm {slow},f_0}$ , then for any invariant ergodic measure $\mu $ there is $\lambda _\mu>1$ such that the unstable Lyapunov exponents of f with respect to $\mu $ , are $\log \lambda _\mu ,2\log \lambda _\mu ,\ldots ,(r+1) \log \lambda _\mu $ occurring with the same multiplicities as for $f_0$ .

We may apply the above theorem to certain Anosov diffeomorphisms that arise on compact nilmanifolds arising as quotients of products of asymmetric nilpotent groups.

Corollary 1.11. Let N be an asymmetric r-step Carnot rational nilpotent group. Let $\Gamma $ be the lattice and $f_0$ the Anosov automorphism of $M=(N\times N)/\Gamma $ obtained from Example 5.12. Then there is a $C^1$ open neighborhood U of $f_0$ in $\mathrm {Diff}{}^\infty (M)$ such that if $f \in U$ admits a smooth splitting $E^u_f=E^u_{\mathrm {fast},f}\oplus E^u_{\mathrm {slow},f}$ along unstable leaves and $E^u_{\mathrm {slow},f}$ is sufficiently uniformly $C^{r}$ -close along unstable leaves to $E^u_{\mathrm {slow},f_0}$ , then for any invariant ergodic measure $\mu $ there is $\lambda _\mu>1$ such that the unstable Lyapunov exponents of f with respect to $\mu $ are $\log \lambda _\mu ,2\log \lambda _\mu ,\ldots ,(r+1)\log \lambda _\mu $ with the corresponding multiplicities as for $f_0$ .

Using Theorem 1.3, we obtain a similar corollary to the one above, but for an Anosov automorphism of a nilmanifold arising as a quotient of a non-asymmetric nilpotent group.

Corollary 1.12. Let N be the three-dimensional Heisenberg group. Let $\Gamma $ be the lattice and $f_0$ the Anosov automorphism of $M=(N\times N)/\Gamma $ obtained from Example 5.14. Then there is a $C^1$ open neighborhood U of $f_0$ in $\mathrm {Diff}{}^\infty (M)$ such that if $f \in U$ admits a smooth splitting $E^u_f=E^u_{\mathrm {fast},f}\oplus E^u_{\mathrm {slow},f}$ along unstable leaves and $E^u_{\mathrm {slow},f}$ is sufficiently uniformly $C^{1}$ -close along unstable leaves to $E^u_{\mathrm {slow},f_0}$ , then for any invariant ergodic measure $\mu $ the unstable Lyapunov exponents for f are $\log \lambda _1,\log \lambda _2$ and $\log \lambda _1+\log \lambda _2$ for some $\lambda _1>1$ and $\lambda _2>1$ depending on $\mu $ .

1.3 Hyperbolic rank rigidity

A Riemannian manifold M has higher hyperbolic rank if every geodesic $c(t)$ in M has a Jacobi field $J(t)$ that makes sectional curvature ${\kappa \cong -1}$ with $c'(t)$ where $J(t) \neq 0$ . Clearly, all rank-one locally symmetric spaces of negative curvature have higher hyperbolic rank after a trivial rescaling of the metric. We made the following conjecture regarding the converse in our recent paper [CNS20, Conjecture 1.3].

Conjecture 1.13. A closed Riemannian manifold with sectional curvatures $\kappa \geq -1$ has higher hyperbolic rank only if it is locally symmetric.

In that paper, we also proved the special case of Conjecture 1.13 when the sectional curvatures satisfy $-\tfrac 14 \geq \kappa \geq -1$ [CNS20, Theorem 1.1]. Using ergodicity of 2-frame flows, Constantine [Con08, Corollary 1] had already characterized constant curvature manifolds by the hyperbolic rank condition for either odd-dimensional manifolds, just assuming non-positive curvature, or under strong pinching assumptions on the curvature, $-(.93)^2 \geq \kappa \geq -1$ . We refer to [CNS20] for more historical discussion, in particular of Hamenstädt’s hyperbolic rank rigidity theorem when the curvature is bounded above by –1.

The current paper began as a sequel to our previous work on the hyperbolic rank rigidity conjecture [CNS20], and indeed we do achieve new results towards Conjecture 1.13. Along the way, we also advance techniques surrounding the Brin–Pesin asymptotic holonomy group for frame flows. We further establish various versions of local rigidity of geodesic flows near the rank-one locally symmetric ones.

One of the main results of the current paper, arising as an application of the results above, is the following local rigidity result for perturbations of locally symmetric metrics with higher hyperbolic rank.

Theorem 1.14. Let $(M, g_0)$ be a closed quaternionic or octonionic hyperbolic locally symmetric manifold. Then there is an open $C^3$ neighborhood U of $g_0$ such that, for any $g\in U$ , if $(M,g)$ has higher hyperbolic rank and $\kappa _g\ge -1$ then $(M,g)$ is locally symmetric.

By Mostow rigidity, it follows that $(M,g)$ is isometric to $(M,g_0)$ . This of course fails for hyperbolic surfaces. Nevertheless, local rigidity in the real hyperbolic case already follows from our previous paper [CNS20].

Our methods do not give us full local rigidity of higher hyperbolic metrics nearby the complex hyperbolic metric. However, we still obtain equality of the Liouville measure and the Bowen–Margulis measure (the unique measure of maximal entropy) for higher-rank perturbations.

Theorem 1.15. Let $(M,g_0)$ be a closed complex hyperbolic manifold. There is an open neighborhood U of $g_0$ in the $C^3$ topology among $C^\infty $ metrics with the following property: for $g\in U$ with higher hyperbolic rank and sectional curvature $\kappa \ge -1$ , the Liouville measure on $SM$ coincides with the (unique) measure of maximal entropy for the geodesic flow of g on $SM$ .

1.4 Local rigidity of projective actions

Sullivan initiated the study of local rigidity of lattice actions on boundary spheres in [Sul85]. There have since appeared a number of results of this kind in various contexts (see, for example, [Asa17, Ghy93, Yue95]). We use our methods to establish local rigidity for certain lattice actions on spheres. First recall that the identity component G of the isometry group of a symmetric space acts by diffeomorphisms on its ideal boundary sphere preserving a smooth distribution $E_0$ , the projection of the slow distribution. This action may also be described in an algebraic way as follows. If $P<G$ is any minimal parabolic subgroup, then we may identify the ideal boundary sphere with $G/P$ on which G naturally acts on the left. We will refer to this action by G, or any subgroup of G, as a projective action.

Theorem 1.16. Let $\rho _0:\Gamma \to \operatorname {Diff}{}^\infty (S^{k})$ for $k=4n-1$ (respectively, $k=15$ ) be the projective representation of a cocompact lattice $\Gamma <Sp(n,1)$ (respectively, $\Gamma <F_4^{-20}$ ). Let $\rho :\Gamma \to \operatorname {Diff}{}^\infty (S^{k})$ be a $C^1$ -close perturbation of $\rho _0$ . If $\rho $ preserves a $C^\infty $ distribution E, $C^1$ -close to $E_0$ , then $\rho $ is $C^\infty $ conjugate to $\rho _0$ .

We note that for a sufficiently small $C^1$ perturbation $\rho $ of $\rho _0$ there always exists a distribution E which is $C^0$ -close to $E_0$ by stability of dominated splittings (see §2.1).

2 Background

2.1 Dynamics: dominated splittings

We begin by reviewing some basic definitions and results about dominated splittings. We will be using these in some of the arguments below.

Definition 2.1. Let $V\to X$ be a $C^0$ vector bundle equipped with a continuously varying norm on fibers over a compact smooth manifold X and $\Lambda \subset X$ be an closed invariant subset for a flow $\varphi _t:X\to X$ . We say that $\Lambda $ admits a dominated splitting of index k for a cocycle $\alpha :V\times \mathbb {R} \to V$ over $\varphi _t$ provided:

1. (1) V restricted to $\Lambda $ splits into two continuous subbundles , both invariant under $\alpha _t$ and with $\dim E=k$ ; and

2. (2) there exist $C>0$ and $0<\lambda <1$ such that for any $x\in \Lambda $ ,

   (2.1)

   

Morally, this definition states that any vector not in E converges uniformly exponentially fast to the subbundle F under $\alpha _t$ .

Examples of dominated splittings include:

• • Anosov geodesic flows $\varphi _t$ on X where $V=TX$ , $E=E^{cs}=E^s\oplus E^c$ center (weak) stable distribution, $F=E^u$ strong unstable distribution and $\alpha =d\varphi _t$ ;

• • geodesic flows of locally symmetric manifolds M of non-constant negative curvature restricted to the unstable bundle where $X=SM$ , the unit tangent bundle, $V=E^u$ strong unstable distribution, $E=E^u_{\mathrm {slow}}$ , $F=E^u_{\mathrm {fast}}$ and $\alpha =d\varphi _t$ ;

• • a one-parameter semisimple subgroup $\varphi _t$ of a semisimple Lie group where $X=G/\Gamma $ for a cocompact lattice $\Gamma <G$ , $V=E^u$ strong unstable distribution, $E=E^u_{<\lambda }$ , $F=E^u_{\geq \lambda }$ for any $\lambda $ in the interior of the Lyapunov spectrum and $\alpha =d\varphi _t$ ;

• • geodesic flows of non-constant negatively curved manifolds with higher hyperbolic rank where $V=E^u$ , $E=E^u_{\mathrm {slow}}$ , $F=E^u_{\mathrm {fast}}$ and $\alpha =d\varphi _{\cdot }$ (cf. §6).

An important property of dominated splittings is that they are stable under perturbations (cf. [Sam16, Proposition 2.3] for the case of a derivative of a map and [CP15, Corollary 2.8] applied to the time-one map of a flow). The proofs of the following proposition were written for the case of tangent bundles and derivative cocycles but apply equally well to the case of general vector bundles and cocycles as in Definition 2.1 (see also [BPS19]).

Proposition 2.2. [CP15, Sam16]

Any sufficiently small $C^0$ perturbation of a cocycle admitting a dominated splitting also admits a dominated splitting of the same index.

2.2 Algebra: varieties of Lie algebras and automorphism groups

The space of n-dimensional Lie algebras can be identified with the space of Lie brackets on $\mathbb {R}^n$ . Thus it is

$$ \begin{align*}\mathcal{L}_n=\{ c\in \operatorname{\mathrm{Hom}}(\Lambda^2\mathbb{R}^n,\mathbb{R}^n)&:c(c(u,v),w)+c(c(v,w),u)+c(c(w,u),v)=0 \\ & \, \, \, \text{ for all } u,v,w\in \mathbb{R}^n \}.\end{align*} $$

Since the defining condition is a quadratic polynomial in the structure coefficients of the bracket c relative to the standard basis of $\mathbb {R}^n$ , $\mathcal {L}_n$ is an affine subvariety of $\operatorname {\mathrm {Hom}}(\Lambda ^2\mathbb {R}^n,\mathbb {R}^n)$ . The space of n-dimensional nilpotent Lie algebras is

$$ \begin{align*}\mathcal{NL}_n=\{ c\in \mathcal{L}_n: c(c(\ldots c(c(u_1,u_2),u_3),\ldots),u_n)=0 \text{ for all } u_1,\ldots,u_n\in \mathbb{R}^n \}.\end{align*} $$

This is an algebraic affine subvariety of $\mathcal {L}_n$ since the defining condition is polynomial in the structure coefficients.

Two Lie algebras on $\mathbb {R}^n$ are isomorphic if and only if there is general linear transformation of $\mathbb {R}^n$ that maps one Lie bracket to the other Lie bracket. Thus, the moduli space of isomorphism classes of nilpotent Lie algebras of dimension n is $\mathcal {NL}_n/\mathrm {GL}(n,\mathbb {R})$ .

We will need the following in §4.

Theorem 2.3. ([BS64], cf. [Zim84, Theorem 3.1.3])

Orbits of real points of an algebraic group acting on the real points of a real algebraic variety are locally closed in the Hausdorff topology.

As a consequence, the orbits of $\mathrm {GL}(n,\mathbb {R})$ on $\mathcal {NL}_n$ are locally closed and $\mathcal {NL}_n/\mathrm {GL}(n,\mathbb {R})$ is countably separated and hence Hausdorff.

We now discuss the automorphism group of a Lie group. We start with the following proposition.

Proposition 2.4. Let G be a connected and simply connected Lie group with Lie algebra $\mathfrak {g}$ . Then the automorphism group $\mathrm {Aut}(G)$ is naturally isomorphic as a Lie group to $\mathrm {Aut}(\mathfrak {g})$ and the latter is a real algebraic variety in $\mathrm {GL}(\mathfrak {g})$ .

Proof. For connected G, $\mathrm {Aut}(G)$ is a closed subgroup of $\mathrm {Aut}(\mathfrak {g})$ with Lie algebra $\mathrm {Der}(\mathfrak {g})$ and these coincide when G is simply connected (cf. [Var84, Theorem 2.75]). Picking a basis of the Lie algebra, a Lie algebra automorphism is completely determined by its associated matrix with respect to this basis. The bracket relations on basis elements give us quadratic polynomial relations in these matrix coefficients. Moreover, any matrix satisfying these relations induces an automorphism. Hence, these polynomial relations provide $\mathrm {Aut}(\mathfrak {g})$ with the structure of a real affine variety in $\mathrm {GL}(\mathfrak {g})\subset \mathbb {P}_{\mathbb {R}}^{(\dim \mathfrak {g})^2}$ .

A nilpotent Lie algebra $\mathfrak {g}$ is graded if it admits a decomposition $\mathfrak {g}=\bigoplus _{i=0}^r \mathfrak {g}^i$ where $[\mathfrak {g}^i,\mathfrak {g}^j]\subseteq \mathfrak {g}^{i+j}$ for all $i,j \geq 0$ with $i+j\leq r$ and $[\mathfrak {g}^i,\mathfrak {g}^j]=0$ otherwise. We call a nilpotent Lie algebra $\mathfrak {g}$ Carnot if it is graded and $[\mathfrak {g}^0,\mathfrak {g}^j] = \mathfrak {g}^{j+1}$ for all $0\leq j\leq r-1$ . A nilpotent Lie group G is called graded (respectively, Carnot) if its Lie algebra is graded (respectively, Carnot). We also call an automorphism of a graded Lie algebra $\mathfrak {g}$ graded if it respects the grading. Finally, an automorphism of a connected graded nilpotent group G is graded if its corresponding automorphism in $\mathrm {Aut}(\mathfrak {g})$ is graded.

Proposition 2.5. Let G be a connected and simply connected graded nilpotent Lie group with Lie algebra $\mathfrak {g}=\bigoplus _{i=0}^r \mathfrak {g}^i$ . Then the graded automorphism group $\mathrm {Aut}_g(G)$ is isomorphic as a Lie group to the graded automorphism group $\mathrm {Aut}_g(\mathfrak {g})$ of the Lie algebra and the latter is a real algebraic variety in $\bigoplus _{i=0}^r \mathrm {GL}(\mathfrak {g}^i)<\mathrm {GL}(\mathfrak {g})$ . In particular, $\mathrm {Aut}_g(\mathfrak {g})$ is a real algebraic group.

Proof. It is clear that automorphisms preserving the grading are a subgroup of automorphisms lying in $\bigoplus _{i=0}^r \mathrm {GL}(\mathfrak {g}^i)<\mathrm {GL}(\mathfrak {g})$ . It is easy to check that the subgroup $\mathrm {Aut}_g(\mathfrak {g})<\mathrm {Aut}(\mathfrak {g})$ corresponds to the subgroup $\mathrm {Aut}_g(G)<\mathrm {Aut}(G)$ under the natural isomorphism given by the previous proposition. It remains to show that $\mathrm {Aut}_g(\mathfrak {g})$ is a subvariety of $\mathrm {Aut}(\mathfrak {g})$ . However, as the graded subspaces are linear, the additional defining equations for the corresponding matrix elements are linear relations. Hence $\mathrm {Aut}_g(\mathfrak {g})$ is also a real algebraic subvariety. Lastly, groups which are real algebraic subvarieties of an algebraic group are themselves algebraic groups.

When G is a simply connected graded nilpotent Lie group we can define a one-parameter group of automorphisms $\delta _s:G\to G$ called the dilations whose associated Lie algebra automorphisms $\delta _s:\mathfrak {g}\to \mathfrak {g}$ are the linear maps given by $\delta _s(x)=s^i x$ for all $x\in \mathfrak {g}^i$ . We denote the group of all dilations by $\mathcal {D}=\{ \delta _s: s\in \mathbb {R}_+ \}$ and observe that it is isomorphic to the multiplicative group $\mathbb {R}_+$ . The dilations are always in the center of $\mathrm {Aut}_g(\mathfrak {g})$ since they are multiples of the identity on each level.

If $\mathfrak {g}$ is a Carnot Lie algebra then any automorphism in $\mathrm {Aut}_g(\mathfrak {g})<\bigoplus _{i=0}^r \mathrm {GL}(\mathfrak {g}^i)$ is completely determined by its component in $\mathrm {GL}(\mathfrak {g}^0)$ . Consequently, we may identify $\mathrm {Aut}_g(\mathfrak {g})$ with a subgroup of $\mathrm {Aut}_g(\mathfrak {g}^0)$ and write $\mathrm {Aut}_g(\mathfrak {g})=L \mathcal {D}$ where $L<\mathrm {SL}_{\pm }(\mathfrak {g}^0)$ . (Here $\mathrm {SL}_{\pm }(\mathfrak {g}^0)$ is the subgroup of $\mathrm {GL}(\mathfrak {g}^0)$ with elements of determinant $\pm 1$ .) Henceforth we set $d_i=\dim \mathfrak {g}^i$ for $i=0,\ldots ,r$ .

Definition 2.6. We call a graded nilpotent Lie group G, and its Lie algebra $\mathfrak {g}$ , asymmetric if its graded automorphism group has the form $\mathrm {Aut}_g(\mathfrak {g})=L\mathcal {D}$ where L belongs to a compact subgroup of $\mathrm {SL}_{\pm }(d_0,\mathbb {R})$ .

Recall that a surjective map $f:X\to Y$ between metric spaces is a homothety if $d(f(x),f(y))=C d(x,y)$ for some constant $C>0$ and every $x,y\in X$ . If a graded nilpotent Lie group G is Carnot, then we may equip G with a left-invariant Carnot–Carathéodory metric arising from the right invariant distribution corresponding to $\mathfrak {g}^0$ equipped with any inner product. In this case, the dilations $\delta _s$ are homotheties as they simply act by the appropriate scalar multiple of the identity on each level preserving the bracket relations. While other non-trivial automorphisms may be homotheties, for example those that restrict to an isometry on the first level, in general they need not all be homotheties. The next lemma shows that being asymmetric is equivalent to $\mathrm {Aut}_g(\mathfrak {g})$ consisting of homotheties for some Carnot–Carathéodory metric.

Lemma 2.7. Let $\mathfrak {g}$ be a Carnot Lie algebra equipped with a right invariant Carnot–Carathéodory metric. If $\mathrm {Aut}_g(\mathfrak {g})$ consists entirely of homotheties, then $\mathfrak {g}$ is asymmetric. Conversely, if $\mathfrak {g}$ is an asymmetric Carnot Lie algebra then $\mathrm {Aut}_g(\mathfrak {g})$ consists of homotheties with respect to some right invariant Carnot–Carathéodory metric.

Proof. If $\mathrm {Aut}_g(\mathfrak {g})=L\mathcal {D}$ consists of homotheties of a Carnot–Carathéodory metric, then $L<\mathrm {SL}_{\pm }(\mathfrak {g}^0)$ and each element of L is uniquely determined by its action on $\mathfrak {g}^0$ . Moreover, each element of L must be an isometry with respect to the given inner product $\langle \cdot ,\cdot \rangle _0$ on $\mathfrak {g}^0$ . Hence L belongs to the compact group $\mathrm {O}(d_0,\langle \cdot ,\cdot \rangle _0)$ .

Conversely, if $\mathrm {Aut}_g(\mathfrak {g})=L\mathcal {D}$ where $L<K$ for some compact subgroup $K<\mathrm {SL}_{\pm }(d_0,\mathbb {R})$ , then K is conjugate into $\mathrm {O}(d_0,\mathbb {R})$ . Equivalently, K belongs to $\mathrm {O}(d_0,\langle \cdot ,\cdot \rangle _0)$ for the inner product on $\mathfrak {g}^0$ conjugate to the standard one. This inner product then induces a Carnot–Carathéodory metric for which $\mathrm {Aut}_g(\mathfrak {g})$ are homotheties.

Asymmetric Carnot nilpotent groups may at first appear to be somewhat special. However, there are many examples and these algebras are generic in many situations.

Examples 2.8. The following classes of graded nilpotent Lie groups G are asymmetric.

1. (1) [Pan89, Proposition 13.1] G belongs to a certain Zariski open dense subset of all 2-step groups with grading $\mathfrak {g}=\mathfrak {g}^0\bigoplus \mathfrak {g}^1$ where $\dim \mathfrak {g}^0\geq 10$ and is even and $3\leq \dim \mathfrak {g}^1 \leq 2 \dim \mathfrak {g}^0-4$ ,

2. (2) [Pan89, Proposition 10.1] G is the (2-step) maximal unipotent subgroup of the isometry group of $\mathbb {H}_{\mathbb {H}}^n$ or $\mathbb {H}_{\mathbb {O}}^2$ for any $n\geq 2$ ,

3. (3) [Pan89, §13] G has an exceptional $(r+1)$ -step filiform algebra ( $r\geq 3$ ) of even dimension $r+3$ given by $\mathfrak {g}=\bigoplus _{i=0}^{(r+1)}\mathfrak {g}^i$ with $\mathfrak {g}^0=\operatorname {\mathrm {span}}\{ y_0,z_0 \}$ and $\mathfrak {g}^i=\operatorname {\mathrm {span}}\{ y_{i} \}$ for $1\leq i\leq r+1$ with bracket relations $[z_0,y_i]=y_{i+1}$ and $[y_i,y_{r-i}]=(-1)^i y_{r+1}$ for $0\leq i\leq r$ and all other brackets $0$ . (The filiform algebras are the graded algebras with minimal possible dimensions of each grading stratum; there are two in each even dimension and one in each odd dimension [Ver70]. However the non-exceptional ones admit non-trivial unipotent graded automorphisms.)

4. (4) [LDOW14, Theorem 1.2] any G with $\mathrm {Aut}(G)=\mathcal {D}$ . Such groups are called ultrarigid nilpotent groups. There are examples of these in at least dimensions 16 and 17 [LDOW14, Examples 3.4 and 3.5].

We do not know if asymmetry is always a generic property of Carnot nilpotent groups with respect to the Zariski topology in sufficiently high dimensions. It it is an open condition in the Hausdorff topology (see Corollary 2.10).

We observe that the underlying space of all Lie algebras $\mathfrak {g}\in \mathcal {NL}_n$ is $\mathbb {R}^n$ . In particular, we have $\mathrm {Aut}{\mathfrak {g}}<\mathrm {GL}(\mathfrak {g})=\mathrm {GL}(n,\mathbb {R})$ . We will thus use the topology of pointed Hausdorff convergence on (closed) subgroups of $\mathrm {GL}(n,\mathbb {R})$ which by abuse of notation we call the ‘Hausdorff topology’. In this topology, a sequence of groups $G_i$ converges to G if the Hausdorff distance $d_H(G_i\cap B(1,r),G\cap B(1,r))$ tends to $0$ as $i\to \infty $ for each $r>0$ , where $B(1,r)$ is the ball of radius r around the identity in $\mathrm {GL}(n,\mathbb {R})$ with respect to the operator norm.

We now show that automorphism groups vary semicontinuously in this topology for nilpotent G.

Proposition 2.9. Let $(G_k)$ be a sequence of connected n-dimensional graded nilpotent Lie groups whose Lie algebras converge to the Lie algebra $\mathfrak {g}$ of G in $\mathcal {NL}_n$ . Then the corresponding graded automorphism groups $Aut_g(\mathfrak {g}_k)$ converge to a Lie subgroup of $Aut_g(\mathfrak {g})<\mathrm {GL}(\mathfrak {g})$ with respect to the topology of pointwise convergence.

Proof. By continuity of the defining polynomial relations satisfied by the matrices in $\mathrm {Aut}_g(\mathfrak {g}_k)$ , any element of the limit will satisfy the polynomial relations required to be in $\mathrm {Aut}_g(\mathfrak {g})$ . Let $H\subset \mathrm {Aut}_g(\mathfrak {g})$ be the collection of limit points. By definition this is a closed subset of $\mathrm {Aut}_g(\mathfrak {g})$ . Considered as equivalence classes of convergent sequences, it is elementary to verify that the product structure given by $[\{ a_n \}]*[\{ b_n \}]=[\{ a_n b_n \}]$ is well defined and gives H a group structure compatible with that of $\mathrm {Aut}_g(\mathfrak {g})$ . Hence H is a Lie subgroup of $\mathrm {Aut}_g(\mathfrak {g})$ .

As a corollary, we deduce that the property of being asymmetric is stable in the variety of nilpotent Lie groups/algebras.

Corollary 2.10. Let G be a connected and simply connected Carnot Lie group of dimension n. If G is asymmetric then there is a neighborhood $\mathcal U$ of G in the variety of n-dimensional Carnot Lie groups such that $\mathcal U$ consists of asymmetric groups.

Proof. Let $(\mathfrak {g}^k)$ be a sequence of Carnot Lie algebras converging to $\mathfrak {g}$ in $\mathcal {NL}_n$ . Write $\mathrm {Aut}_g(\mathfrak {g})=L\mathcal {D}$ where $L=\mathrm {Aut}_g(\mathfrak {g})\cap \mathrm {SL}_{\pm }(d_0,\mathbb {R})$ lies in some compact group K. Without loss of generality, we may assume after a conjugation that $L<K=\mathrm {O}(d_0,\mathbb {R})$ . Similarly, let $\mathrm {Aut}_g(\mathfrak {g}^k)=L_k\mathcal {D}$ for $L_k=\mathrm {Aut}_g(\mathfrak {g}^k)\cap \mathrm {SL}_{\pm }(d_0,\mathbb {R})$ .

By Proposition 2.9, the $L_k$ converge to a subgroup of L. We wish to show that $L_k$ belongs to a compact subgroup. We will first consider $L_k^0$ , the connected component of the identity of $L_k$ . We fix a norm on $\mathbb {R}^n$ . If the $L_k^0$ are not contained in a compact group, then the operator norm, $\lVert \cdot \rVert _{\mathrm {op}}$ is unbounded on $L_k^0$ . Then $L_k^0$ admits a path whose elements have norms taking on all values in $[1,\infty )$ , and in particular there is an element $a_k$ with $\lVert a_k \rVert _{\mathrm {op}}=2$ . The sequence $\{ a_k \}$ admits a convergent subsequence to an element of determinant $\pm 1$ , but which does not have norm one, leading to a contradiction.

By Proposition 2.5, $\mathrm {Aut}_g(\mathfrak {g}^k)$ and thus $L_k$ are algebraic groups. An algebraic group has only finitely many connected components by [Whi57, Theorem 3]. Therefore $L_k$ must lie in a compact subgroup.

3 Tangent cones of sub-Riemannian manifolds

In this section we analyze the structure of tangent cones and relate the derivatives of smooth maps preserving the horizontal distribution to the Carnot derivatives. Under suitable assumptions, this leads to resonances and even arithmeticity of the Lyapunov spectrum.

Throughout this section we will be exploiting the properties of sub-Riemannian manifolds equipped with the generic and horizontal distribution as defined in §1.1. Toward this end, we first recall the notion of a tangent cone.

Definition 3.1. Let $(X,d)$ be a metric space and $x\in X$ . We call any pointed Gromov–Hausdorff limit of $(X,p, td)$ when $t\to \infty $ a tangent cone of X at p.

In general, tangent cones of metric spaces at a point may not exist, and when they do, they may not be unique. However, in the setting of sub-Riemannian manifolds, Mitchell [Mit85] showed that they exist, and Margulis and Mostow [MM00] proved that they are unique.

Theorem 3.2. ([Mit85, Theorem 1], [MM00])

Let $(N,d_C)$ be a sub-Riemannian manifold. Suppose that the metric $d_C$ comes from a smooth horizontal distribution E. If the distribution E is generic in a neighborhood of $p\in N$ then the tangent cone of N at p exists, is unique and is a Carnot nilpotent Lie group with a right-invariant Carnot–Carathéodory metric induced from the distribution of homogeneous degree-one vector fields tangent to E with the inner product inherited from E.

The very last statement of the above theorem is not stated explicitly in the cited references, but follows from the construction as will be explained in §3.1. There we will also define the notion of homogeneous vector fields.

We next discuss the construction and structure of this nilpotent group and its relationship to the tangent cone at each point $p\in N$ (cf. [Mit85, §3]). We will need these details for our application to sub-Riemannian dynamics in §4.

3.1 Carnot metrics and the Métivier correspondence

Métivier developed a correspondence between iterated brackets of horizontal vector fields in a neighborhood of a given point $p\in N$ and certain homogeneous vector fields possessing nilpotent bracket relations. We present here a compressed summary of the details.

Let $X_i$ , $i=1,\ldots ,d_0=\dim E$ , be smooth vector fields on N that are tangent to and form a basis for the horizontal distribution in a neighborhood U of a point $p\in N$ . After possibly shrinking the neighborhood U, among the commutators of the $X_i$ we may choose an ordered subset $Y_1,\ldots ,Y_n$ that form a basis of $T_yN$ at each point $y\in U$ and such that $Y_1=X_1,\ldots ,Y_{d_0}=X_{d_0}$ , and together with the next ordered subcollection ( $Y_{d_0+1},\ldots ,Y_{d_0+d_1}$ ) form a basis for the first commutator subbundle $E^1(y)$ , and so on. We call such a moving frame a graded basis. We sometimes index this basis according to the level as $(Y_{0,1},\ldots ,Y_{r,d_r})$ .

For any smooth vector field X on N, let $s\mapsto \exp s X$ be the one-parameter subgroup of diffeomorphisms acting on N induced by the flow of X, that is, along integral curves of X. For each y in the neighborhood U of p the map $y=\exp (\sum _i y_i Y_i)(p)\mapsto (y_i)\in \mathbb {R}^n$ is a diffeomorphism from U to some neighborhood of $0$ in $\mathbb {R}^n$ . We call the collection of components $y_i$ normal coordinates on U. We emphasize that normal coordinates do not define (commuting) coordinates in the ordinary sense since the distribution is not integrable.

For a generic horizontal distribution E, we define the partition floor function to be $[i]_E:=k+1$ if $\dim E^{k-1}(x)<i\leq \dim E^k(x)$ for $k=0,\ldots ,r$ and where we adopt the convention that $\dim E^{-1}=0$ . When the distribution E is understood we simply write $[i]$ instead of $[i]_E$ . We define a dilatation $\delta _s:U\to N$ for $s\in [0,\infty )$ in terms of normal coordinates by taking $\delta _s(y_i)=s^{[i]}y_i$ . The $\delta _s$ are local diffeomorphisms for $s>0$ .

For a vector field of the form $X=y_{i_1}^{\alpha _1}\cdots y_{i_k}^{\alpha _k}Y_j$ , written in terms of the fields $Y_j$ and their associated normal coordinate functions $y_j$ , we define the degree of X to be $[j]-\sum _{m=1}^k \alpha _m [i_m]$ . By taking the maximum degree over monomial terms we can extend the notion of degree to vector fields of the form $X=\sum _i p_i(y_1,\ldots ,y_n) Y_i$ for polynomials $p_i$ . Finally, we can extend this notion to arbitrary smooth vector fields in a neighborhood of p by considering the degree of vanishing at p (see [MM00, §3] for details). A vector field is homogeneous of degree i if each of its component terms has degree exactly i. We can decompose each vector field X into its homogeneous components. If X has degree at most q and $X^{(q)}$ denotes the homogeneous degree-q part of X at p, and similarly Y has degree at most s at p and $Y^{(s)}$ is its homogeneous degree-s part, then by [MM00, §3] the bracket $[X,Y]$ has degree at most $q+s$ at p and, moreover,

(3.1) $$ \begin{align} [X,Y]^{(q+s)}=[X^{(q)},Y^{(s)}]. \end{align} $$

For each horizontal vector field X on a neighborhood of p, we obtain a homogeneous degree-one vector field $\widehat {X}_p=\lim \nolimits _{t\to \infty } t^{-1}(\delta _t)_*(X)$ on N (see [ Mét76 , equation (3.2), p. 487]). The vector field $\widehat {X}_p$ is precisely the homogeneous degree-one part at p of the horizontal vector field X. Moreover, the following lemma shows that if $X_1$ and $X_2$ are two horizontal fields with $X_1(p)=X_2(p)$ , then $\widehat {X}_{1;p}=\widehat {X}_{2;p}$ .

Lemma 3.3. If $X_1,\ldots ,X_k$ is a basis of horizontal vector fields in a neighborhood of $p\in N$ for a sub-Riemannian manifold N, and $X=\sum _{i=1}^{d_0} a_i X_i$ is any horizontal vector field on N, then $\widehat {X}_p=\sum _{i=1}^{d_0} a_i(p) \widehat {X}_{i;p}$ .

Proof. Since X is horizontal it has degree at most one, and we can write it in the basis as $X(y)=\sum _i a_i(y) X_i$ for functions $a_i$ on the neighborhood U of p. Express this as $X(y)= \sum _i a_i(p)X_i+ \sum _i (a_i(y)-a_i(p)) X_i$ . Since the functions $a_i(y)-a_i(p)$ vanish at p, they have degree at most $-1$ at p and the $X_i$ have degree $1$ . Hence $\sum _i (a_i(y)-a_i(p)) X_i$ has degree at most $0$ and $\widehat {X}_p=\sum _i a_i(p)\widehat {X}_{i;p}$ .

Métivier [ Mét76 , Théorème 3.1] showed that the homogeneous vector fields $\widehat {X}_{i;p}$ generate a nilpotent Lie algebra through taking commutators, and Mitchell [Mit85, Lemmas 3.1 and 3.2] showed that the corresponding simply connected nilpotent group with its corresponding Carnot–Carathéodory metric is isometric and isomorphic to the tangent cone at p. By Lemma 3.3, for any horizontal field X we have that $\widehat {X}_p$ is in the span of $\widehat {X}_{i;p}$ .

The homogeneous degree-one part $\widehat {X}_p$ of a horizontal field X thus corresponds to a right invariant field $\overline {X}$ on $TC_pN$ . In other words, $\overline {X}$ belongs to the Lie algebra of $TC_pN$ , which we denote by $\mathfrak {n}_p=\operatorname {Lie}(TC_pN)$ . Let $\mathfrak {n}^0_p\subset \mathfrak {n}_p$ be the subspace spanned by the set of all $\overline {X}$ , and inductively set $\mathfrak {n}^i_p=[\mathfrak {n}^{0}_p,\mathfrak {n}^{i-1}_p]\subset \mathfrak {n}_p$ for $i\in \{ 1,\ldots ,r \}$ . The degree of homogeneity of the vector fields provides a natural grading for the Lie algebra $\mathfrak {n}_p$ as pointed out in [MM95, §8.2] and [MM00, §4], furnishing $\mathfrak {n}_p$ with a Carnot nilpotent structure.

Among the k-fold commutators of $\widehat {X}_{i;p}$ we may choose a set of vector fields $\widehat {X}_{k,j;p}$ whose corresponding right invariant fields $\overline {X}_{k,j}$ form a basis of the kth commutator space $\mathfrak {n}^{k}_p\subset \mathfrak {n}_p$ of $\mathfrak {n}^0_p$ respecting the grading. In particular, we may choose indices so that $\widehat {X}_{0,j;p}=\widehat {X}_{j;p}$ . This provides an isomorphism of Lie algebras, $\mathcal {M}_p: \operatorname {span}\{ \widehat {X}_{0,1;p},\ldots ,\widehat {X}_{r,d_r;p} \}\to \mathfrak {n}_p$ , where the domain is a Lie algebra of vector fields under the ordinary bracket. We name this identification between linear combinations of the $\{ \widehat {X}_{k,j;p} \}$ and elements of $\mathfrak {n}_p$ the Métivier correspondence at p. Note that this correspondence depends on the choice of graded basis. By Lemma 3.3, the Métivier correspondence induces a linear inclusion $\imath :E(p)\to \mathfrak {n}_p$ with image $\mathfrak {n}^0_p$ by $X(p)\mapsto \overline {X}=\mathcal {M}_p(\widehat {X}_p)$ . We will omit the p subscript on vector fields when the base point of the tangent cone is understood.

Following [MM00], define an equivalence class on curves of M by $\alpha \sim \beta $ if and only if

$$ \begin{align*} \lim\limits_{s\to 0} \frac{1}{s}d_C(\alpha(s),\beta(s))=0. \end{align*} $$

The tangent cone $TC_pN$ can be considered as the space of equivalence classes of curves $[\alpha ]$ with $\alpha (0)=p$ which are equivalent to a dilatation orbit curve at p, $[\delta _s y]$ for some $y\in N$ , with the distance

(3.2) $$ \begin{align} d([\alpha], [\beta]) = \lim_{s\to 0} s^{-1} d_C(\alpha(s),\beta(s)). \end{align} $$

The above interpretation generalizes a standard interpretation of the tangent space to a differentiable manifold as the space of equivalence classes of differentiable curves.

In the next proposition we will consider the subspace topology on $\mathcal {NL}_n\subset \operatorname {\mathrm {Hom}}(\Lambda ^2\mathbb {R}^n,\mathbb {R}^n)$ instead of the coarser Zariski topology of the variety.

Proposition 3.4. Let $(N,d_C)$ be a smooth sub-Riemannian manifold. Suppose that the metric $d_C$ comes from a generic smooth horizontal distribution E. Then the metrics, nilpotent structures, and the corresponding isomorphism classes, of the tangent cones of N vary continuously on N.

Proof. We need only show that the isomorphism class of the tangent cones vary continuously locally. By Theorem 3.2, the nilpotent structures depend only on the horizontal distribution, not the choice of basis above used to construct the structure. Hence we may choose the initial basis freely. We now refer back to the notation at the beginning of §3.1.

For a neighborhood U of a point $p\in N$ , by the genericity assumption, we may choose the basis of fields $X_i$ for $i=1,\ldots ,d_0$ so that the commutators of $X_i$ in $E^k$ do not degenerate to $E^{k-1}$ at any point of U if they do not do so at the point p. In particular, the same indices can be used to choose the same fields $Y_i$ , for $i=1,\ldots ,n$ at each point $x\in U$ . Therefore, the construction of all the $\widehat {X}_{k,j;x}$ varies smoothly in a neighborhood of p. This implies that the structure constants of the Lie algebras of the tangent cones vary smoothly on U and hence this implies that the nilpotent structure varies continuously on the space of nilpotent structures described in §2.2. This continuity thus passes to the orbit quotient space of nilpotent isomorphism classes as well.

Lastly, since the metric on $TC_xN$ is a Carnot–Carathéodory metric for the distributions induced by the inclusion of $E(x)$ in $\mathfrak {n}_x$ , the metric also varies continuously on U, and thus on all of N.

Remark 3.5. Observe that continuity in the subspace topology on $\mathcal {NL}_n$ equipped with metrics coming from the inner product on E as it varies in an open set implies continuity in the pointed Gromov–Hausdorff topology as it implies convergence on metric nets.

By Theorem 3.2, the metric on $TC_pN$ is a Carnot–Carathéodory metric for the right invariant distribution induced from $\mathfrak {n}^0_p\subset \mathfrak {n}_p$ . Next, we identify the norm on the distribution that gives rise to the metric.

Lemma 3.6. The Carnot–Carathéodory metric on $TC_pN$ is induced by the right invariant distribution corresponding to $\mathfrak {n}^0_p\subset \mathfrak {n}_p$ equipped with the Riemannian inner product coming from $E(p)$ under the above identification.

We will denote the norm on $E(p)$ by $\lVert \cdot \rVert _p$ and that on $\mathfrak {n}^0_p$ by $\lVert \cdot \rVert _{\mathfrak {n}^0_p}$ .

Proof. Given a unit-speed sub-Riemannian geodesic segment $\gamma $ starting at p, let X be its tangent field extended to a smooth horizontal field on the neighborhood U of p. By [MM00, Proposition 5.6], the Carnot–Carathéodory metric induced on $TC_pN$ does not depend on the choice of basis vector fields (satisfying certain assumptions). Using X as a coordinate field, we obtain

$$ \begin{align*} d_C(p,\delta_s\exp(X)(p))=d_C(p,\exp(s X)(p))=d_C(p,\gamma(s))=s. \end{align*} $$

This implies in $TC_pN$ that $d([p],[\delta _s \exp (X)(p)])=1$ by (3.2). Since the identity element in $TC_pN$ is the class $[p]$ of the constant curve, and $\widehat {X}=X$ along its integral curve, $[\delta _s \exp (t X)(p)]=e^{t\overline {X}}$ and therefore $d(1,e^{t \overline {X}})=t$ . The induced norm on the first level $\mathfrak {n}^0_p$ of the Lie algebra then gives $\|\overline {X}\|_{\mathfrak {n}^0_p}=1$ .

Thus the identification preserves the norm of vectors. Since inner products are uniquely determined by their norms, the inner products are preserved as well.

3.2 Carnot derivatives of maps preserving horizontal distributions

Next we recall the notion of the Carnot derivative which will be crucial to our purposes. This object was first introduced by Pansu in the setting of nilpotent Lie groups in [Pan89] based on earlier work of Mostow on quasiconformal maps of boundaries of rank-one symmetric spaces [Mos73]. We will need the following version due to Margulis and Mostow in the setting of sub-Riemannian manifolds.

Definition 3.7. [MM95, Definition 10.3.1]

Let $(N,d_C)$ and $(N',d_C')$ be sub-Riemannian manifolds, and let $f:(N,d_C)\to (N',d_C')$ be a map. For every $p\in N$ , we denote by $f_t: (N,p,td_C)\to (N',f(p),td_C')$ the induced map for every $t>0$ . The function f is called Carnot differentiable at p if the sequence $f_t$ converges uniformly as $t\to \infty $ on compact sets to a map between tangent cones at p and at $f(p)$ . And we call f Carnot differentiable if it is Carnot differentiable at every point.

In the next few propositions we explore the relationship between the Carnot derivative and the ordinary derivative.

Proposition 3.8. Let $f:(N,d_C)\to (N',d_C')$ be a $C^1$ local diffeomorphism between sub-Riemannian manifolds generated by generic horizontal distributions E and $E'$ . If f preserves the distributions, that is, $Df(E)=E'$ , then f is Carnot differentiable everywhere. Moreover, the Carnot derivative $f_*$ is an isomorphism between the corresponding tangent cones.

Proof. Since the statement is local, it is enough to consider any sufficiently small precompact neighborhood U of an arbitrary point $p\in N$ where f is a diffeomorphism and we use instead of f. We assume U is chosen such that there exist smooth horizontal vector fields $X_1,\ldots ,X_{d_0}$ on U which form a basis for $E(x)$ at each $x\in U$ . We let $X_i'=Df(X_i)$ for each $i=1,\ldots ,d_0$ . By hypothesis, $\{ X_1',\ldots ,X_{d_0}' \}$ forms a basis of $E'(y)$ at each $y\in U'$ . As above, these vector fields give us normal coordinates and we use them to define our respective dilatations $\delta _s$ on U and $\delta _s'$ on $U'=f(U)$ .

We note that $f:U\to U'$ is a quasi-conformal diffeomorphism since f has uniformly bounded derivatives on the compact set $\overline {U}$ . By [MM95, Theorem 10.5] and its proof, f induces an isomorphism of tangent cones at each point x belonging to a certain full measure set $U^0\cap f^{-1}(\Omega _{X_1'}\cap \cdots \cap \Omega _{X_{d_0}'})$ to be defined shortly. (Note that the $f^{-1}$ was inadvertently left off of the $\Omega _{X_i'}$ in the proof of [MM95].) We will show that both $U^0$ and $f^{-1}(\Omega _{X_i'})$ for any $i=1,\ldots ,d_0$ are all of U in our case.

The definition of $U^0$ given in [MM95, §10.1] simplifies in our situation. It consists of the density points for the Hausdorff measure of $d_C$ (cf. [MM95, Definition 2.1.4]) for the set A where the Lipschitz constant of f is uniformly bounded. However, since f is $C^1$ with uniformly bounded derivatives, $A=U$ . Moreover, every point $x\in U$ is a density point since U is open.

We now recall the definition of the $\Omega _{X_i'}$ from [MM95]. For $i=1,\ldots , d_0$ , the right invariant vector fields $\overline {X}_i'\in \mathfrak {n}_y$ on $TC_yU'$ correspond to the homogeneous degree-one fields $\widehat {X}_{i;y}'$ , namely $\overline {X}_i'=\mathcal {M}_y(\widehat {X}_{i;y}')$ . Recall that the $X_i'$ were used to define the normal coordinates and the dilatation $\delta _s'$ . Hence, along the integral curve of $X_i'$ through y, we have $1/{t} (\delta _t')_* X_i'=X_i'$ , and hence $\widehat {X}_{i;y}'=\lim \nolimits _{t\to \infty } 1/{t} (\delta _t')_* X_i'=X_i'$ along the same curve. Let $e^{s \overline {X}_i'}$ denote the one-parameter subgroup in $TC_y U'$ of the right invariant field $\overline {X}_i'$ , viewed as an element of the Lie algebra $\mathfrak {n}_y$ . Furthermore, following [MM95, §8.6], let $\epsilon _y$ denote the homeomorphism of a neighborhood of $1\in TC_y U'$ to $U'$ which takes the equivalence class of the path ${\{ \delta _s' z \}}_{s\in [0,1]}$ in $TC_y U'$ to the point $z\in U'$ . (The equivalence classes of the germs as $s\to 0$ of the paths $\delta _s' z$ correspond to points in the tangent cone by [Mit85, Lemmas 3.1 and 3.2].)

As in [MM95], for $i=1,\ldots ,d_0$ let $A_{X_i'}$ be the set consisting of all points $y\in U'$ such that $s^{-1} d_C'((\exp s X_i')(y),\epsilon _y(e^{s \overline {X}_i'}))$ tends to $0$ uniformly as $s\to 0$ . Finally, the set $\Omega _{X_i'}$ consists of all the density points in $A_{X_i'}$ for the Hausdorff measure of $d_C'$ .

Finally, observe that since $\widehat {X}_{i;y}'=X_i'$ along its integral curve through y, we have

$$ \begin{align*} \delta'_t \exp(s X_i')(y)=\delta'_t \exp(s \widehat{X}_{i;y}')(y). \end{align*} $$

The latter curve in s converges in the pointwise Gromov–Hausdorff sense through the metric spaces $(U',y,({d'}/{t}))$ to the curve $e^{s\overline {X_i'}}\in TC_yN'$ as $t\to 0$ . In other words, the curve $\epsilon _y(e^{s \overline {X_i'}})$ coincides with the curve $\exp (s X_i')(y)$ for every $y\in U'$ . In particular, $A_{X_i'}=U'$ and therefore $\Omega _{X_i'}=U'$ since every point of $U'$ is a density point for the Hausdorff measure.

Diffeomorphisms preserving the horizontal distribution E share certain properties with their corresponding automorphisms. To investigate this relationship we will need to make certain correspondences more explicit.

Definition 3.9. Let $f:N\to N'$ be a diffeomorphism of sub-Riemannian manifolds sending the horizontal distribution E to the horizontal distribution $E'$ . Assume that E is generic at a point $p\in N$ . For a graded basis $Y_1,\ldots ,Y_n$ of coordinate vector fields on N around p (respectively, $Y_1',\ldots ,Y_n'$ around $f(p)$ ) with $X_1=Y_1,\ldots ,X_{d_0}=Y_{d_0}$ horizontal, let $\mathcal {M}_p:\operatorname {span}\{ \widehat {X}_{0,1},\ldots ,\widehat {X}_{r,d_r} \}\to \mathfrak {n}_p$ be the induced Métivier correspondence. We then define

$$ \begin{align*} \widehat{f}_*:\operatorname{span}\{ \widehat{X}_{0,1},\ldots,\widehat{X}_{r,d_r} \}\to \operatorname{span}\{ \widehat{X'}_{0,1},\ldots,\widehat{X'}_{r,d_r} \} \end{align*} $$

by setting $\widehat {f}_*(Z)=\mathcal {M}_{f(p)}^{-1}\circ df_* \circ \mathcal {M}_p(Z)$ where $df_*:\mathfrak {n}_p\to \mathfrak {n'}_{f(p)}$ is the Lie derivative of the isomorphism $f_*:TC_pN\to TC_{f(p)}N'$ .

Lemma 3.10. Let $f:N\to N'$ be a local $C^\infty $ diffeomorphism of sub-Riemannian manifolds N and $N'$ sending the horizontal distribution E of N to the horizontal distribution $E'$ of $N'$ . Assume that E is generic at a point $p\in N$ , and let $f_*:TC_pN\to TC_{f(p)}N'$ be the induced isomorphism. For any horizontal vector field X we have

$$ \begin{align*} \widehat{f}_*\widehat{X}=\widehat{Df(X)}. \end{align*} $$

Moreover, if $({X}_{0,1},\ldots ,{X}_{r,d_r})$ is a graded basis around p then $\widehat {f}_*[\widehat {X}_{i,k},\widehat {X}_{j,l}]=[\widehat {f}_*\widehat {X}_{i,k},\widehat {f}_*\widehat {X}_{j,l}]$ .

Proof. By Proposition 3.8 the limit of the maps $f_t$ given in Definition 3.7 exists and is $f_*:TC_pN\to TC_{f(p)}N'$ .

The push-forward under f of a horizontal vector field X is also horizontal and therefore converges under metric rescaling to a right invariant horizontal field $\overline {Df(X)}$ by [MM95, Proposition 8.10]. As shown in the proof of [MM95, Theorem 10.5], the image under $f_*$ of the one-parameter subgroup $e^{s\overline {X}}$ is the one-parameter subgroup of this limit push-forward field. In other words, for all $g\in TC_pN$ and $s\in \mathbb {R}$ , we have

$$ \begin{align*} f_*(e^{s \overline{X}}g)=e^{s \overline{Df(X)}}f_*(g). \end{align*} $$

Taking Lie derivatives, we have $df_*(\overline {X})=\overline {Df(X)}$ and therefore under the Métivier correspondence we have $\widehat {f}_*\widehat {X}=\widehat {Df(X)}$ as claimed.

The second statement follows directly from the Métivier correspondence and (3.1) and the fact that ${df}_*[\overline {X}_{i,k},\overline {X}_{j,l}]=[{df}_*\overline {X}_{i,k},{df}_*\overline {X}_{j,l}]$ .

By Lemma 3.6 the inclusion $\imath :E(p)\hookrightarrow \mathfrak {n}_p$ defined earlier is an isometry onto its image $\mathfrak {n}^0_p$ with respect to the Riemannian norm and the norm inducing the Carnot–Carathéodory metric. The following corollary gives further relationships between some of the objects defined thus far.

Corollary 3.11. In the setting of Proposition 3.8, the following diagram commutes:

where $\mathbf {exp}$ is the Lie exponential map $\mathbf {exp}(X)=e^X$ . Moreover, for all $\xi \in E(p)$ , we have $\lVert D_pf(\xi ) \rVert _{f(p)}=\lVert df_*(\imath \xi ) \rVert _{\mathfrak {n'}^0_{f(p)}}$ .

Proof. The lower part of the diagram commutes as a standard consequence of Lie theory.

By Lemma 3.10, we have $\widehat {f}_*\widehat {X}=\widehat {Df(X)}$ . Since $X(p)=\widehat {X}_p(p)\in E(x)$ for a horizontal field X, we have

$$ \begin{align*} df_*(\imath(X(p))&=df_*(\overline{X})=df_*\mathcal{M}_p(\widehat{X})=\mathcal{M}_{f(p)}\widehat{f}_*\widehat{X}=\mathcal{M}_{f(p)}\widehat{Df(X)}\\&=\overline{Df(X)}=\imath'(D_{p}f(X(p))), \end{align*} $$

where the third equality is by the definition of $\widehat {f}_*$ . This establishes commutativity of the upper diagram.

The final statement follows from the commutativity of the diagram and the fact that the inclusions are isometric.

The following relationship between the differential of a local $C^\infty $ diffeomorphism at a fixed point and its induced automorphism will prove to be crucial. A priori there is no clear connection between the Carnot derivative and the ordinary derivative of a smooth sub-Riemannian map. However, we are able to establish such a connection in some important special cases.

Proposition 3.12. Let $f:N\to N$ be a local $C^\infty $ diffeomorphism of a sub-Riemannian manifold N preserving the horizontal distribution E. Assume that E is generic at a point $p\in N$ where $f(p)=p$ , and let $f_*:TC_pN\to TC_pN$ be the induced automorphism. If is diagonalizable with real eigenvalues $(\lambda _1,\ldots ,\lambda _{d_0})$ , then there exists a basis of $T_pN$ such that $D_pf:T_pN\to T_pN$ is upper triangular with diagonal values $(\lambda _1,\ldots ,\lambda _{d_0},\lambda _{i_1}\lambda _{i_2},\ldots ,\lambda _{i_1}\lambda _{i_2}\lambda _{i_3},\ldots )$ .

Proof. Again let $U\subset N$ be a small neighborhood around the point p and $X_1,\ldots , X_n$ be a graded basis where $X_1,\ldots , X_{d_0}$ are horizontal vector fields spanning E on U. As above, normal coordinates define a dilatation $\delta _t:U\to N$ . Then if X and Y are horizontal, by Métivier [ Mét76 ], X and Y are vector fields of degree at most $1$ at p and $\widehat {X}=\lim \nolimits _{t\to \infty }{1}/{t}(\delta _t)_* X$ is the homogeneous degree-one part of X. Note that $[X, Y]$ is a vector field of degree at most $2$ and we denote the homogeneous degree-two part by $[X,Y]^{(2)}=\lim \nolimits _{t\to \infty }{1}/{t^2}(\delta _t)_*[ X, Y]$ .

Without loss of generality we may assume that fields $X_1,\ldots ,X_{d_0}$ in the chosen basis have corresponding right invariant fields $\overline {X}_i$ which diagonalize $f_*$ on the first level of the Lie algebra $\mathfrak {n}$ . Indeed, by assumption, there are $\lambda _i\in \mathbb {R}$ such that $df_*\overline {X_i}=\lambda _i \overline {X_i}$ for $i=1,\ldots ,d_0$ .

By Corollary 3.11, for $i=1,\ldots ,d_0$ we have $\imath D_pf(X_i(p))=df_*(\imath (X_i(p)))=df_*(\overline {X}_i)=\lambda _i \overline {X}_i$ . Taking $\imath ^{-1}$ , we have $D_pf (X_i(p)) = \lambda _i X_i(p)$ .

By Lemma 3.10, if $[X_i,X_j]$ is a graded basis element, then $\widehat {f}_*[\widehat {X}_i,\widehat {X}_j]=[\widehat {f}_*\widehat {X}_i,\widehat {f}_*\widehat {X}_j]$ . Therefore we have

$$ \begin{align*} df_*\mathcal{M}_p[\widehat{X}_i,\widehat{X}_j]&=\mathcal{M}_p[\widehat{f}_*\widehat{X}_i,\widehat{f}_*\widehat{X}_j], \\ df_*[\mathcal{M}_p\widehat{X}_i,\mathcal{M}_p\widehat{X}_j]&=\mathcal{M}_p[\widehat{Df {X}_i},\widehat{Df {X}_j}], \\ [df_*\overline{X}_i,df_*\overline{X}_j]&=\mathcal{M}_p[{Df {X}_i},{Df {X}_j}]^{(2)},\\ \lambda_i\lambda_j[\overline{X}_i,\overline{X}_j]&=\mathcal{M}_p[{Df {X}_i},{Df {X}_j}]^{(2)}. \end{align*} $$

It then follows that

(3.3) $$ \begin{align} [{Df {X}_i},{Df {X}_j}]^{(2)} &=\lambda_i\lambda_j\mathcal{M}_p^{-1}[\overline{X}_i,\overline{X}_j]=\lambda_i\lambda_j[\widehat{X}_i,\widehat{X}_j]=\lambda_i\lambda_j[X_i,X_j]^{(2)}. \end{align} $$

By hypothesis $\{ X_1,\ldots ,X_{d_0} \}\cup \{ [X_i,X_j]:1\le i,j \le d_0 \}$ spans $E^1$ . Since the homogeneity degree of $Df[X_i,X_j]=[Df(X_i),Df(X_j)]$ and $[X_i,X_j]$ at p is at most $2$ , ${Df[X_i,X_j]-\lambda _i\lambda _j[X_i,X_j]}$ has homogeneity degree at most $1$ . Evaluating (3.3) at p, we have $[Df(X_i),Df(X_j)](p)-\lambda _i\lambda _j[X_i,X_j](p)\in E(p)=E^0(p)$ .

Continuing by induction, if $X_{i}$ and $X_{j}$ are graded basis fields of degree at most q and s respectively such that $[X_{i},X_{j}]$ is a basis field, then a nearly identical computation shows

$$ \begin{align*} [{Df {X}_i},{Df {X}_j}]^{(q+s)}=\sigma_i\sigma_j[X_i,X_j]^{(q+s)}, \end{align*} $$

where by the inductive hypothesis $\sigma _i=\lambda _{i_1}\cdots \lambda _{i_\kappa }$ and $\sigma _j=\lambda _{j_1}\cdots \lambda _{j_\tau }$ for some indices $i_1,\ldots ,i_\kappa $ and $j_1,\ldots ,j_\tau $ .

This shows that in the $\{ X_i(p) \}$ basis, $Df$ is upper triangular. If, restricted to the horizontal distribution, the diagonal values are $\lambda _1,\ldots ,\lambda _{d_0}$ , then other diagonal elements are products of the $\lambda _i$ as claimed.

Now we are equipped to prove Theorem 1.4 in the introduction.

Theorem 1.4. Let $f:N\to N$ be a local $C^\infty $ diffeomorphism of a sub-Riemannian manifold N. Assume that the horizontal distribution is generic at a point $p\in N$ where $f(p)=p$ , and that the graded automorphism $f_*:TC_pN\to TC_pN$ induced from f is a homothety. Then for some $\lambda>1$ , has Lyapunov exponents $\log \lambda ,2\log \lambda ,\ldots ,(i+1)\log \lambda $ with corresponding multiplicities $d_0=\dim \mathfrak {n}^0,\ldots ,d_i=\dim \mathfrak {n}^{i}$ for $i=0,\ldots ,r$ .

Proof. Let $({X}_{0,1},\ldots ,{X}_{r,d_r})$ be a graded basis of N in a neighborhood of p. As before, we let $\displaystyle {\mathcal {M}_p: \operatorname {span}\{ \widehat {X}_{0,1},\ldots ,\widehat {X}_{r,d_r} \}\to \mathfrak {n}_p}$ be the Métivier correspondence. We will show that with respect to the basis $({X}_{0,1}(p),\ldots ,{X}_{r,d_r}(p))$ , the matrix for $D_pf$ is a block upper triangular matrix. Suppose that $f_*=\delta _{\lambda } h$ , where $\delta _\lambda $ is the dilation of scale $\lambda>0$ and h is an isometry of the Carnot–Carathéodory metric of $TC_pN$ which necessarily lies in a compact subgroup of the graded automorphism group.

By Lemma 3.10, $\widehat {f}_*\widehat {X}_{0,j}=\widehat {Df(X_{0,j})}$ . Thus,

$$ \begin{align*}\widehat{Df(X_{0,j})}=\mathcal{M}_p df_* \mathcal{M}_p^{-1}\widehat{X}_{0,j}=\mathcal{M}_p df_* \overline{X}_{0,j}=\lambda\mathcal{M}_p dh \overline{X}_{0,j}=\lambda\mathcal{M}_p dh \mathcal{M}_p^{-1}\widehat{X}_{0,j}.\end{align*} $$

Letting $\widehat {dh}=\mathcal {M}_p dh \mathcal {M}_p^{-1}$ , we can rewrite the above equation as $\widehat {Df(X_{0,j})}=\lambda \widehat {dh}\widehat {X}_{0,j}$ . Since $dh$ is a graded Lie algebra homomorphism, $\widehat {dh}$ is also a graded Lie algebra homomorphism of $ \operatorname {span}\{ \widehat {X}_{0,1},\ldots ,\widehat {X}_{r,d_r} \}$ . In particular, $\widehat {dh}$ preserves $ \operatorname {span}\{ \widehat {X}_{i,1},\ldots ,\widehat {X}_{i,d_i} \}$ for every $i=0,\ldots , r$ . It follows that $\widehat {dh}\widehat {X}_{0,j}\in \operatorname {span}\{ \widehat {X}_{0,1},\ldots ,\widehat {X}_{0,d_0} \}$ for every ${j=1, \ldots , d_0}$ . Therefore, $D_pf(\widehat {X}_{0,j}(p))=\widehat {Df(X_{0,j})}(p)=(\widehat {dh}\widehat {X}_{0,j})(p)\in \operatorname {span}\{ \widehat {X}_{0,1}(p),\ldots , \widehat {X}_{0,d_0}(p) \}=E(p)$ .

Consider $X_{1,l}$ . Suppose that $X_{1,l}=[X_{0,i},X_{0,j}]$ . By Lemma 3.10, $\widehat {f}_*\widehat {X_{1,l}}=\widehat {f}_*[\widehat {X}_{0,i},\widehat {X}_{0,j}] =[\widehat {f}_*\widehat {X}_{0,i},\widehat {f}_*\widehat {X}_{0,j}]=[\widehat {DfX}_{0,i},\widehat {DfX}_{0,j}]$ . And thus,

(3.4) $$ \begin{align} (Df[{X}_{0,i},{X}_{0,j}])^{(2)}=[{DfX}_{0,i},{DfX}_{0,j}]^{(2)}=[\widehat{DfX}_{0,i},\widehat{DfX}_{0,j}]=\lambda^2\widehat{dh}\widehat{X_{1,l}}. \end{align} $$

Set $V^{(i)}(p)=\operatorname {span}\{ {X}_{i,1}(p),\ldots ,{X}_{i,d_i}(p) \}$ so that $E^{i}(p)=V^{(0)}(p)\oplus \cdots \oplus V^{(i)}(p)$ . We note that $\widehat {dh}\widehat {X}_{0,l}\in \operatorname {span}\{ \widehat {X}_{0,1},\ldots ,\widehat {X}_{0,d_0} \}$ since $\widehat {dh}$ is a graded Lie algebra homomorphism. We observe that for every $l=1\ldots ,d_1$ , we have $\widehat {X}_{1,l}(p) \in \operatorname {span}\{ X_{1,l}(p), X_{0,1}(p), \ldots , X_{0,d_0}(p) \}$ . It follows that $(Df[{X}_{0,i},{X}_{0,j}])^{(2)}(p)\in \operatorname {span}\{ \widehat {X}_{0,1}(p),\ldots ,\widehat {X}_{0,d_0}(p), \widehat {X}_{1,1}(p),\ldots ,\widehat {X}_{1,d_1}(p) \}=E^{1}(p)$ . On the other hand,

$$ \begin{align*}(Df[{X}_{0,i},{X}_{0,j}])(p)=(Df[{X}_{0,i},{X}_{0,j}])^{(2)}(p)+(Df[{X}_{0,i},{X}_{0,j}])^{(\le 1)}(p),\end{align*} $$

where $(Df[{X}_{0,i},{X}_{0,j}])^{(\le 1)}$ is the homogeneous part of $(Df[{X}_{0,i},{X}_{0,j}])$ of degree at most 1. Evaluating a vector field of degree at most 1 at p, we get a vector in $E(p)= \operatorname {span}\{ \widehat {X}_{0,1}(p),\ldots ,\widehat {X}_{0,d_0}(p) \}$ . Therefore, $(Df[{X}_{0,i},{X}_{0,j}])(p)\in \operatorname {span}\{ \widehat {X}_{0,1}(p),\ldots , \widehat {X}_{0,d_0}(p),\widehat {X}_{1,1}(p),\ldots ,\widehat {X}_{1,d_1}(p) \}=E^{1}(p)$ .

Inductively, for $X_{m+1,l}=[X_{m,i},X_{0,j}]$ , we obtain

(3.5) $$ \begin{align} (Df[{X}_{m,i},{X}_{0,j}])^{(m+2)}&=[{DfX}_{m,i},{DfX}_{0,j}]^{(m+2)}\nonumber\\&=[({DfX}_{m,i})^{(m+1)},\widehat{DfX}_{0,j}]=\lambda^{m+2}\widehat{dh}\widehat{X}_{m+1,l}. \end{align} $$

And it follows that

(3.6) $$ \begin{align} \begin{split} (D_pf)(({X}_{m+1,l})(p))&=\lambda^{m+2}(\widehat{dh}\widehat{X}_{m+1,l})(p)+(Df{X}_{m+1,l})^{(\le m+1)}(p) \in E^{m+1}(p), \end{split} \end{align} $$

where the notation $Y^{(\le m+1)}$ indicates the sum of the homogeneous parts of any vector field Y of degree at most $m+1$ .

Therefore, with respect to the basis $({X}_{0,1}(p),\ldots ,{X}_{r,d_r}(p))$ , $D_pf$ is a block upper triangular matrix. We let $A_i$ be the blocks on the diagonal of $D_pf$ corresponding to the basis elements $\{ X_{i,1}(p),\ldots ,X_{i,d_i}(p) \}$ , for every $i=0,\ldots , r$ . We note that the size of $A_i$ is $\dim (\mathfrak {n}^{i}_p)\times \dim (\mathfrak {n}^{i}_p)$ . Since $(Df{X}_{m+1,l})^{(\le m+1)}(p)\in E^{m}(p)$ , equation (3.6) implies that

$$ \begin{align*}\operatorname{\mathrm{proj}}_{V^{(m+1)}(p)}(D_pf)(({X}_{m+1,l})(p))=\operatorname{\mathrm{proj}}_{V^{(m+1)}(p)}(\lambda^{m+2}(\widehat{dh}\widehat{X}_{m+1,l})(p)),\end{align*} $$

where $\operatorname {\mathrm {proj}}_{V^{(m+1)}(p)}$ is the projection of $T_pN$ onto $V^{(m+1)}(p)$ along the subspace $\check {V}^{(m+1)}(p):=V^{(0)}(p)\oplus \cdots \oplus V^{(m)}(p)\oplus V^{(m+2)}(p)\oplus \cdots \oplus V^{(r)}(p)$ . Thus the $(j,l)$ entry of the block matrix $A_{m+1}$ is the $X_{m+1,j}(p)$ component of $\lambda ^{m+2}(\widehat {dh}\widehat {X}_{m+1,l})(p)$ , for $j,l\in \{ 1,\ldots ,d_{m+1} \}$ .

Next, we estimate Lyapunov exponents of block matrices $A_0, \ldots , A_r$ . Applying equation (3.5) for $f^n$ instead of f, we obtain

(3.7) $$ \begin{align} (Df^n[{X}_{m,i},{X}_{0,j}])^{(m+2)}&=[{Df^nX}_{m,i},{Df^nX}_{0,j}]^{(m+2)}\nonumber\\ &=[({Df^nX}_{m,i})^{(m+1)},\widehat{Df^nX}_{0,j}]=\lambda^{n(m+2)}\widehat{dh}^n\widehat{X}_{m+1,l}. \end{align} $$

Since

$$ \begin{align*} (Df^n[{X}_{m,i},{X}_{0,j}])(p)=(Df^n[{X}_{m,i},{X}_{0,j}])^{(m+2)}(p)+(Df^n[{X}_{m,i},{X}_{0,j}])^{(\le m+1)}(p) \end{align*} $$

and

$$ \begin{align*} (Df^n[{X}_{m,i},{X}_{0,j}])^{(\le m+1)}(p)\in E^{m}(p), \end{align*} $$

the projection of $(Df^n[{X}_{m,i},{X}_{0,j}])(p)$ along $\check {V}^{(m+1)}(p)$ onto $V^{(m+1)}(p)$ is the same as the projection of $\lambda ^{n(m+2)}\widehat {dh}^n\widehat {X}_{m+1,l}(p)$ to $V^{(m+1)}(p)$ along $\check {V}^{(m+1)}(p)$ .

We claim that

(3.8) $$ \begin{align} \operatorname{span}\{ \widehat{X}_{m+1,1}(p),\ldots,\widehat{X}_{m+1,k_{m+1}}(p) \}\cap \check{V}^{(m+1)}(p)=\{0\}. \end{align} $$

Indeed, we have $\operatorname {span}\{ \widehat {X}_{0,1}(p),\ldots ,\widehat {X}_{i,d_{i}}(p) \}= \operatorname {span}\{ {X}_{0,1}(p),\ldots ,{X}_{i,d_{i}}(p) \}$ for all $i=0,\ldots ,m+1$ . Moreover, these sets of vectors form bases of their corresponding spanned vector spaces. This implies the claim.

This claim implies that for every $w\neq 0$ in $\operatorname {span}\{ \widehat {X}_{m+1,1}(p),\ldots ,\widehat {X}_{m+1,d_{m+1}}(p) \}$ , its projection to $V^{(m+1)}(p)$ along $\check {V}^{(m+1)}(p)$ is non-zero. For every $v\neq 0$ in $V^{(m+1)}(p)$ , there are $c_1,\ldots , c_{d_{m+1}}\in \mathbb {R}$ not all being zero such that $v=c_1 {X}_{m+1,1}(p) +\cdots + c_{d_{m+1}}{X}_{m+1,d_{m+1}}(p)$ . We then have

$$ \begin{align*} D_pf^nv&=c_1 (Df^n{X}_{m+1,1})(p) +\cdots + c_{k_{m+1}}(D_pf^n{X}_{m+1,k_{m+1}}(p))\\ &=c_1 (Df^n{X}_{m+1,1})^{(m+2)}(p) +\cdots + c_{k_{m+1}}(D_pf^n{X}_{m+1,k_{m+1}}^{(m+2)}(p))\\ &\quad+ c_1 (Df^n{X}_{m+1,1}^{(\le m+1)})(p) +\cdots + c_{k_{m+1}}(D_pf^n{X}_{m+1,k_{m+1}}^{(\le m+1)})(p)\\ &=\lambda^{n(m+2)}\widehat{dh}^n (c_1 \widehat{X}_{m+1,1}(p) +\cdots + c_{k_{m+1}}\widehat{X}_{m+1,k_{m+1}}(p))\\ &\quad+ c_1 (Df^n{X}_{m+1,1}^{(\le m+1)})(p) +\cdots + c_{k_{m+1}}(D_pf^n{X}_{m+1,k_{m+1}}^{(\le m+1)})(p). \end{align*} $$

By same argument as above, $D_pf^nv$ and $\lambda ^{n(m+2)}\widehat {dh}^n (c_1 \widehat {X}_{m+1,1}(p) +\cdots + c_{d_{m+1}}\widehat {X}_{m+1,d_{m+1}} (p))$ have the same projection onto $\operatorname {span}\{ {X}_{m+1,1}(p),\ldots ,{X}_{m+1,d_{m+1}}(p) \}$ along $\operatorname {span}\{ {X}_{1,1}(p),\ldots ,{X}_{m,d_{m}}(p) \}$ . Since $c_1, \dots, c_{d_{m+1}}$ are not all zero, the vector $c_1 \widehat {X}_{m+1,1}(p) +\cdots + c_{d_{m+1}}\widehat {X}_{m+1,d_{m+1}}(p)$ is non-zero, and thus has a non-zero projection on $\operatorname {span}\{ {X}_{m+1,1}(p),\ldots ,{X}_{m+1,d_{m+1}}(p) \}$ by (3.8).

Since h is in a compact subgroup of $\mathrm {GL}(n,\mathbb {R})$ , there is a subsequence $(n_k)$ such that $\widehat {dh}^{n_k}$ converges to the identity. Choose an arbitrary norm $\|\cdot \|$ on $V^{(m+1)}(p)$ . Therefore, we may find two constants $C_1,C_2>0$ independent of k such that

$$ \begin{align*}C_1\lVert v \rVert<\|\operatorname{\mathrm{proj}}_{V^{(m+1)}(p)}(\widehat{dh}^{n_k}(v))\|<C_2\lVert v \rVert.\end{align*} $$

Hence,

$$ \begin{align*}C_1\lambda^{n_k(m+2)}\|v\|<\|\operatorname{\mathrm{proj}}_{V^{(m+1)}(p)}(D_pf^{n_k}v)\|<C_2\lambda^{n_k(m+2)}\|v\|,\end{align*} $$

for every $v\neq 0$ in $V^{(m+1)}(p)$ . It follows that $A_{m+1}$ has all Lyapunov exponents equal to $(m+2)\log \lambda $ .

Finally, as is well known, the Lyapunov exponents of a block upper triangular matrix are the same as those of the corresponding block diagonal matrix.

If $f_*$ fails to be a homothety, we are still able to recover a system of inequalities among the Lyapunov exponents. This is Theorem 1.2 from the introduction.

Theorem 1.2. Let $f:N\to N$ be a local $C^\infty $ diffeomorphism of a sub-Riemannian manifold N. Assume that the horizontal distribution is generic at a point $p\in N$ where $f(p)=p$ . Suppose the tangent cone $TC_pN$ is $(r+1)$ -step and set $n_{-1}=0$ and $n_i=\dim E^{i}(p)$ for $i\in \{ 0,\ldots ,r \}$ . If has Lyapunov exponents $(\log \lambda _1,\log \lambda _2,\ldots ,\log \lambda _{n_0},\log \lambda _{{n_0}+1},\ldots ,\log \lambda _{n_1},\ldots ,\log \lambda _{n_{i}})$ listed with multiplicity and with $\log \lambda _{n_{j-1}+1},\ldots ,\log \lambda _{n_{j}}$ in non-decreasing order for each $j\in \{ 0,\ldots ,i \}$ , then for each $i\in \{ 0,\ldots , r \}$ and $j\in \{ n_{i-1}+1,\ldots ,n_{i} \}$ we have $(i+1) \log \lambda _1\leq \log \lambda _{j}\leq (i+1)\log \lambda _{n_0} $ .

Proof. By switching f with $f^{-1}$ we observe that it is sufficient to prove just the lower bound. Let $\{ {X}_{0,1},\ldots ,{X}_{r,d_r} \}$ be a graded basis of N in a neighborhood of p. We let $\displaystyle {\mathcal {M}_p: \operatorname {span}\{ \widehat {X}_{0,1},\ldots ,\widehat {X}_{r,d_r} \}\to \mathfrak {n}_p}$ be the Métivier correspondence. Note that $n_i=\sum _{j=0}^i d_i$ for $i=0,\ldots ,r$ .

Our assumptions imply that the induced map $f_*$ exists as a graded automorphism. Its derivative $df_*$ and the corresponding $\widehat {f}_*=\mathcal {M}_{f(p)}^{-1}\circ df_* \circ \mathcal {M}_p$ are graded Lie algebra automorphisms. With respect to the basis $(\widehat {X}_{0,1},\ldots ,\widehat {X}_{r,d_r})$ , $\widehat {f}_*$ is a block diagonal matrix. We further assume that we have chosen the initial horizontal basis $X_{0,1},\ldots ,X_{0,d_0}$ so that $\widehat {f}_*$ is in real Jordan canonical form with respect to $\widehat {X}_{0,1},\ldots ,\widehat {X}_{0,d_0}$ .

We note that for every $0\le j\le r$ , $\operatorname {span}\{ \widehat {X}_{0,1}(p) \ldots ,\widehat {X}_{j,d_j}(p) \}=\operatorname {span}\{ {X}_{0,1}(p),\ldots , {X}_{j,d_j}(p) \}\subset T_pN$ . Since $\widehat {f}_*$ is a block diagonal matrix, $(\widehat {f}_*\widehat {X}_{j,l})(p)\in \operatorname {span}\{ \widehat {X}_{j,1}(p), \ldots ,\widehat {X}_{j,d_j}(p) \}$ for every $1\le l\le d_j$ . On the other hand, $(Df{X}_{j,l} - \widehat {f}_*\widehat {X}_{j,l})(p)\in \operatorname {span}\{ \widehat {X}_{0,1}(p), \ldots ,\widehat {X}_{j-1,d_{j-1}}(p) \}$ by Lemma 3.10. Here we used the convention that $\operatorname {span}\{ \widehat {X}_{0,1}(p), \ldots ,\widehat {X}_{j-1,d_{j-1}}(p) \}=\{0\}$ when $j=0$ . By replacing f by $f^n$ , we have that $(Df^n{X}_{j,l} - \widehat {f}_*^n\widehat {X}_{j,l})(p)\in \operatorname {span}\{ \widehat {X}_{0,1}(p), \ldots ,\widehat {X}_{j-1,d_{j-1}}(p) \}$ for every $n\in \mathbb {N}$ . In particular, in the $\widehat {X}_{j,l}$ basis, $D_pf$ is block upper triangular and with diagonal blocks coinciding with the matrix of $\widehat {f}_*$ in this basis. Because $\widehat {f}_*^n\widehat {X}_{j,l}(p)\in \operatorname {span}\{ \widehat {X}_{j,1}(p),\ldots ,\widehat {X}_{j,d_j}(p) \}$ and $\operatorname {span}\{ \widehat {X}_{j,1}(p),\ldots ,\widehat {X}_{j,d_j}(p) \}\cap \operatorname {span}\{ \widehat {X}_{0,1}(p),\ldots ,\widehat {X}_{j-1,d_{j-1}}(p) \}=\{0\}$ , the growth rate of $Df^nX_{j,l}(p)=D_pf(X_{j,l}(p))$ is at least as large as the growth rate of $\widehat {f}_*^n\widehat {X}_{j,l}(p)$ for all $1\leq l \leq d_j$ . Hence for any $v=\sum _{i=1}^{d_j} a_i {X}_{j,1}(p)$ the growth rate of $Df^n v$ is at least as large as the growth rate of $\widehat {f}_*^n(w)$ where $w=\sum _{i=1}^{d_j} a_i \widehat {X}_{j,1}(p)$ . By the block upper triangular structure of $D_pf$ , it follows that the growth rate of $D_pf^n(v)$ is at least as large as the minimum of the growth rates of $\widehat {f}_*^n$ restricted to $\operatorname {span}\{ \widehat {X}_{j,1}(p), \ldots , \widehat {X}_{j,d_j}(p) \}$ .

The conclusion now follows from the fact that the growth rate of $\widehat {f}_*^n\widehat {X}_{j,i}(p)$ is at least $j+1$ times the minimum of growth rates of $\widehat {f}_*^n\widehat {X}_{0,1}(p), \ldots , \widehat {f}_*^n\widehat {X}_{0,d_0}(p)$ due to the bracket relations. (Recall from the choice of initial basis that $\widehat {f}_*$ achieves its minimal growth rate on one of the vectors $\widehat {X}_{0,1},\ldots ,\widehat {X}_{0,d_0}$ .) Since ${X}_{0,1},\ldots ,{X}_{0,d_0}$ are horizontal, $\widehat {f}_*^n\widehat {X}_{0,i}(p)=D_pf^n({X}_{0,i}(p))$ . Thus, the minimum of growth rates of $\widehat {f}_*^n\widehat {X}_{0,1}(p), \ldots , \widehat {f}_*^n\widehat {X}_{0,d_0}(p)$ is at least $\log \lambda _1$ .

Remark 3.13. Note also that in the ordering of the Lyapunov exponents given in Theorem 1.2, we may have $\lambda _{d_i+1}<\lambda _{d_i}$ . We note that since $D_pf^n(X_{j,l}(p))= \widehat {f}_*^n\widehat {X}_{j,l}(p)+ Y_n$ for some $Y_n\in \operatorname {span}\{ \widehat {X}_{0,1}(p),\ldots ,\widehat {X}_{j-1,d_{j-1}}(p) \}$ , the iterates of the $Y_n$ component may grow faster than $\widehat {f}_*^n\widehat {X}_{j,l}(p)$ except when $l=d_j$ . Hence we may not have $(i+1)\log \lambda _2\leq \log \lambda _{j}$ for any $j\in \{ n_i+1,\ldots ,n_{i+1} \}$ .

Remark 3.14. We note that Proposition 3.12 and Theorem 1.4 may be combined and slightly generalized as follows. Suppose $df_*:\mathfrak {n}_p\to \mathfrak {n}_p$ has the form $df_*=D\cdot h$ where D is diagonalizable with positive diagonal values $(\lambda _1,\ldots ,\lambda _k)$ when restricted to $\mathfrak {n}^{0}_p$ and h is an isometry that preserves the eigenspaces of D. Then we obtain Lyapunov exponents of $d_pF$ of the form $\sum _{j=0}^{k} \log {\lambda _{i_j}}$ for $k\leq r$ and certain combinations of indices $i_j$ .

Next we present the proof of Theorem 1.3.

Theorem 1.3. Let $f:N\to N$ be a local $C^\infty $ diffeomorphism of a sub-Riemannian manifold N. Assume that the horizontal distribution E is generic at a point $p\in N$ where $f(p)=p$ . Suppose the tangent cone $TC_pN$ is isomorphic to the Heisenberg group $H^{2n+1}$ . If $\log \lambda _1, \log \lambda _2, \ldots , \log \lambda _{2n}$ are the Lyapunov exponents of listed with multiplicity and $\log \lambda _{2n+1}$ is the remaining Lyapunov exponent of $D_pf$ , then $\log \lambda _1+\log \lambda _2+\cdots +\log \lambda _{2n}=n \log \lambda _{2n+1}$ .

Proof. The Heisenberg group $H^{2n+1}$ has a distinguished basis of right invariant vector fields, $\overline {X}_1,\ldots , \overline {X}_n, \overline {Y}_1, \ldots , \overline {Y}_n, \overline {Z}$ such that $[\overline {X}_i,\overline {Y}_j]=\delta _{ij}\overline {Z}$ and $[\overline {X}_i,\overline {X}_j]=[\overline {Y}_i,\overline {Y}_j]=[\overline {X}_i,\overline {Z}]=[\overline {Z},\overline {Y}_j]=0$ . By assumption, $TC_pN$ is isomorphic to $H^{2n+1}$ . Hence in a neighborhood of p there is a graded basis of fields $X_1,\ldots , X_n, Y_1, \ldots , Y_n,Z=[X_1,Y_1]$ such that $\mathcal {M}_p(\widehat {X}_i)=\overline {X}_i$ , $\mathcal {M}_p(\widehat {Y}_i)=\overline {Y}_i$ and $\mathcal {M}_p([\widehat {X}_i,\widehat {Y}_i])=\overline {Z}$ for $i\in \{ 1,\ldots ,n \}$ . In other words, setting $\widehat {Z}=[\widehat {X}_1,\widehat {Y}_1]$ , we have $[\widehat {X}_i,\widehat {Y}_j]=\delta _{ij}\widehat {Z}$ and $[\widehat {X}_i,\widehat {X}_j]=[\widehat {Y}_i,\widehat {Y}_j]=[\widehat {X}_i,\widehat {Z}]=[\widehat {Z},\widehat {Y}_j]=0$ for all $i,j\in \{ 1,\ldots ,n \}$ . Note that by the identity (3.1), we have $Z^{(2)}=\widehat {Z}$ .

With respect to the chosen basis we have an identification $\mathfrak {n}_p=\mathbb {R}^{2n}\oplus \mathbb {R}$ . The graded automorphism group of $H^{2n+1}$ is $\mathbb {Z}_2\ltimes (\operatorname {Sp}(2n,\mathbb {R}) \times \operatorname {Dil})$ where $\operatorname {Sp}(2n,\mathbb {R})$ is the symplectic group, $\operatorname {Dil}\cong \mathbb {R}_+$ are the dilations and the $\mathbb {Z}_2$ generator switches $\overline {X}_i$ and $\overline {Y}_i$ for $i=1,\ldots n$ [Til70]. Since $df_*:\mathfrak {n}_p\to \mathfrak {n}_p$ is a graded automorphism, there is an $A\in \operatorname {Sp}(2n,\mathbb {R})$ , a block matrix $U=\big[\begin {smallmatrix} 0 & I_n\\ I_n &0\end {smallmatrix}\big]$ or $U=I_{2n}$ , and $\sigma>0$ such that $df_*$ has the form $df_*(x,z)=(\sigma U A x, \sigma ^2z)$ . (Here the U matrix represents the corresponding element of $\mathbb {Z}_2$ .) Since Lyapunov exponents of $f^2$ are twice those of f and $(UA)^2\in \operatorname {Sp}(2n,\mathbb {R})$ , without loss of generality we may replace f with $f^2$ and assume $df_*(x,z)=(\sigma A x, \sigma ^2z)$ . Next we show that with respect to the basis $\{ X_1(p),\ldots , X_n(p),Y_1(p),\ldots ,Y_n(p),Z(p) \}$ of $T_pN$ , $D_pf=\big[\begin {smallmatrix}\sigma A&B\\0&\sigma ^2\end {smallmatrix}\big]$ for some $2n\times 1$ matrix B.

It follows that with respect to the basis $\{\widehat {X}_1,\ldots , \widehat {X}_n, \widehat {Y}_1, \ldots , \widehat {Y}_n\}$ , $\widehat {f}_*(\widehat {X}_i)=\sigma A\widehat {X}_i$ and $\widehat {f}_*(\widehat {Y}_j)=\sigma A\widehat {Y}_j$ . We note that $\widehat {X}_i(p)=X_i(p)$ and $\widehat {Y}_j(p)=Y_j(p)$ . By Lemma 3.10, $\widehat {f}_*(\widehat {X}_i)=\widehat {DfX_i}$ and $\widehat {f}_*(\widehat {Y}_j)=\widehat {DfY_i}$ . Thus with respect to the basis $\{X_1(p),\ldots , X_n(p), Y_1(p), \ldots , Y_n(p)\}$ , by restricting of $D_pf$ on $\operatorname {span}\{ X_1(p),\ldots , X_n(p), Y_1(p), \ldots , Y_n(p) \}$ , we have $D_pf(X_i(p))=AX_i(p)$ and $D_pf(Y_j(p))=AY_j(p)$ .

Moreover, we have

$$ \begin{align*} \sigma^2Z^{(2)}&=\widehat{f}_*(Z^{(2)})=\widehat{f}_*([\widehat{X}_1,\widehat{Y}_1])=[\widehat{f}_*\widehat{X}_i,\widehat{f}_*\widehat{Y}_i]\\&=[\widehat{DfX}_1,\widehat{DfY}_1]=(Df[X_1,Y_1])^{(2)}=(DfZ)^{(2)}. \end{align*} $$

Thus,

$$ \begin{align*} D_pf(Z(p))\kern1.5pt{=}\kern1.5pt DfZ(p)\kern1.5pt{=}\kern1.5pt(DfZ)^{(2)}(p)+(DfZ)^{(\le 1)}(p)\kern1.5pt{=}\kern1.5pt\sigma^2Z^{(2)}(p)+(DfZ) {}^{(\le 1)}(p). \end{align*} $$

We note that $(DfZ)^{(\le 1)}(p)\in \operatorname {span}\{ X_1(p),\ldots , X_n(p), Y_1(p), \ldots , Y_n(p) \}$ . Also, $Z=Z^{(2)}+Z^{(\leq 1)}$ with $Z^{(\leq 1)}(p)\in \operatorname {span}\{ X_1(p),\ldots , X_n(p), Y_1(p), \ldots , Y_n(p) \}$ . Hence, with respect to the basis $\{{X}_1(p),\ldots , {X}_n(p), {Y}_1(p), \ldots , {Y}_n(p), Z(p)\}$ of $T_pN$ , there is a $2n\times 1$ matrix B such that $D_pf=\big[\begin {smallmatrix}\sigma A&B\\0&\sigma ^2\end {smallmatrix}\big]$ .

Now it follows that the Lyapunov exponents of f are $2 \log \sigma $ and the logarithms of the moduli of the eigenvalues of $\sigma A$ . Since $\det (\sigma A)=\sigma ^{2n}$ , we conclude that $n\log \lambda _{2n+1}=2n\log \sigma $ is equal to the sum of all other exponents.

4 Sub-Riemannian dynamics

We will apply the ideas from the previous section to distributions tangent to foliations invariant under flows and diffeomorphisms. We start with a very general and abstract statement which we then develop in the context of hyperbolic dynamics. We will make the following hypotheses throughout this section.

Standing Hypotheses 4.1.

• • X is a smooth compact manifold with $f: X\to X$ a $C^1$ diffeomorphism.

• • $\mathcal {F}$ is an f-invariant foliation tangent to a continuous distribution D.

• • The leaves of $\mathcal F$ are $C^\infty $ and f is $C^\infty $ along leaves.

• • There is a $df$ -invariant continuous subdistribution $E\subseteq D$ .

We define

$$ \begin{align*}\Omega=\{v\in X: E \text{ is } C^\infty, \text{ horizontal and generic in a neighborhood of } v \text{ in } \mathcal{F}(v)\}.\end{align*} $$

Moreover, at each point $v\in \mathcal {F}(v)$ we have a filtration

$$ \begin{align*} E(v)=E^0(v)\subset E^1(v)\subset E^2(v) \subset \cdots \end{align*} $$

of vector subspaces of $T_v\mathcal {F}(v)$ that are formed by taking the span of i-fold brackets of vector fields tangent to E at v. These form well-defined distributions $E^i$ on $\Omega $ and we let $r=r(v)$ be the index such that $E^r(v)=T_v\mathcal {F}(v)$ for $v\in \Omega $ . Observe that r is constant on each connected component of $\mathcal {F}(v)\cap \Omega $ .

We first establish the following generalization of Theorem 1.1.

Theorem 4.2. (Foliated tangent cone structure theorem)

Assume Hypotheses 4.1. Then we have the following statements.

1. (1) $\Omega $ is f-invariant and intersects every leaf of $\mathcal {F}$ in a (possibly empty) open set.

2. (2) For every $v\in \Omega $ , the tangent cone $TC_v\mathcal {F}(v)$ of $\mathcal {F}(v)$ at v exists, is a Carnot nilpotent Lie group with a left invariant Carnot–Carathéodory metric and varies continuously on the intersection of $\Omega $ with each leaf.

3. (3) The map f induces a Lie group isomorphism between the tangent cones $TC_v\mathcal {F}(v)$ of $\mathcal {F}(v)$ at $v\in \Omega $ and $TC_{f(v)}\mathcal {F}(f(v))$ of $\mathcal {F}(f(v))$ at $f(v)$ . We call this the Carnot derivative of f at v.

4. (4) If for a sufficiently large k, the distribution E is uniformly $C^k$ along $\mathcal {F}$ (see Definition 4.4 below), then $\Omega $ is open in X and the tangent cone $TC_v\mathcal {F}(v)$ varies continuously in $v\in \Omega $ .

Similarly, for a $C^1$ flow $\varphi _t: X\to X$ and flow invariant distributions $E\subseteq D$ the analogous statements hold.

Inspecting the proof below, it will be clear that the degree of uniform regularity $C^k$ of E suffices to be one less than the maximum nilpotency degree of the tangent cones on $\Omega $ . These nilpotency degrees are in turn all bounded by the dimension of the leaves of $\mathcal {F}$ minus one.

We will break the proof into two main lemmas.

Lemma 4.3. Assume Hypotheses 4.1. Statements (1)–(3) of Theorem 4.2 hold.

Proof. For every $w\in \mathcal {F}(v)$ , we let $E^i(w)$ be the subspace of $D(w)$ spanned by all brackets of vector fields tangent to $E(w)$ of orders at most i. Let $n_i(w)=\dim E^i(w)$ . Then $n_i(w)$ is f-invariant by the f-relation of brackets ( $df[X,Y]=[df(X),df(Y)]$ ). Moreover, $n_i$ is lower semicontinuous. Indeed, if a bracket of vector fields of order i does not vanish at a point then it does not vanish in a neighborhood in the unstable manifold of that point. The semicontinuity of $n_i$ implies that $n_i$ is locally constant on an open set of each leaf for every i. This latter open subset of each leaf is the intersection of $\Omega $ with that leaf. Because $n_i$ is f-invariant, $\Omega $ is also f-invariant.

For every $v\in \Omega $ , we consider a small enough neighborhood of v in $\mathcal {F}(v)$ . This neighborhood is a sub-Riemannian manifold on which the horizontal distribution E is $C^\infty $ and generic. By Theorem 3.2, the tangent cone $TC_v\mathcal {F}(v)$ exists. The continuity of tangent cones follows from Proposition 3.4. Moreover, the map f is a $C^1$ diffeomorphism from this neighborhood of v in $\mathcal {F}(v)$ onto a neighborhood of $f(v)\in \Omega $ in $\mathcal {F}(f(v))$ . The map f preserves the distribution E, and so by Proposition 3.8, $df$ induces an isomorphism $(df)_*:TC_v\mathcal {F}(v)\to TC_{f(v)}\mathcal {F}(f(v))$ .

The flow case follows by restricting to a single time map $f=\varphi _t$ .

Note that while $\Omega $ is open in each leaf, the entire $\Omega $ may not be open in X. Next, we will discuss the continuity of tangent cones transversely to the foliation $\mathcal {F}$ . We first recall the relevant topology.

Definition 4.4. Assume a foliation $\mathcal {F}$ of X has $C^k$ leaves. A function on X is called (transversely) uniformly $C^k$ along $\mathcal {F}$ if its kth-order derivatives exist along leaves of the foliation and are continuous on X. A distribution is called uniformly $C^k$ along $\mathcal {F}$ if there are local vector fields forming a basis of the distribution that are uniformly $C^k$ along $\mathcal {F}$ .

We complete the proof of Theorem 4.2 with the following lemma.

Lemma 4.5. Assume Hypotheses 4.1. If the distribution E is uniformly $C^k$ along $\mathcal {F}$ for a sufficiently large k, then the set $\Omega $ is open in X and the tangent cone $TC_v\mathcal {F}(v)$ varies continuously in $v\in \Omega $ . In particular, statement (4) of Theorem 4.2 holds.

Proof. Let $v\in \Omega $ . The distribution E is horizontal and generic in a neighborhood of v in $\mathcal {F}(v)$ . Since E is uniformly $C^k$ , a neighborhood of v in X is contained in $\Omega $ and thus $\Omega $ is open.

For the second claim, choose $v_0\in \Omega $ . As in the proof of Proposition 3.4, on any sufficiently small neighborhood $U\subset \Omega $ of $v_0$ we may choose a basis $X_{1},\ldots ,X_{d_0}$ of vector fields spanning E such that:

1. (1) the $X_j$ vary continuously in the $C^k$ topology; and

2. (2) if $\{ (j_1,\ldots ,j_i) \}\subset \{ 1,\ldots ,d_0 \}^i$ is a family of indices whose corresponding iterated brackets $[\widehat {X}_{j_1},[\widehat {X}_{j_2},[\widehat {X}_{j_3},\ldots ]\ldots ]]$ form the chosen basis $\{ \widehat {X}_{i,j} \}$ of $\mathcal {M}_{v_0}^{-1}(\mathfrak {n}_{v_0}^{i-1})$ , then the corresponding brackets also form a basis of $\mathcal {M}_{v}^{-1}(\mathfrak {n}_{v}^{i-1})$ for all $v\in U$ .

Indeed, for any fixed index j the vector fields $\widehat {X}_{j}=\widehat {X}_{j,v}$ vary continuously in v, as do the bracket operations, and thus the non-degeneracy of these iterated brackets is an open condition. It follows that the corresponding nilpotent structures vary continuously, as do the isomorphism classes of the nilpotent tangent cones.

Since the metric on each tangent cone $TC_v\mathcal {F}(v)$ is the Carnot–Carathéodory metric for the distributions induced by the inclusion of $E(v)$ in the Lie algebra $\mathfrak {n}_v$ of $TC_v\mathcal {F}(v)$ , the metric also varies continuously on the neighborhood U, and thus on all of $\Omega $ .

Combining this result with good dynamical properties, we obtain the following corollary.

Corollary 4.6. Assume Hypotheses 4.1. If the distribution E is uniformly $C^k$ for a sufficiently large k, then the isomorphism class of tangent cones $TC_v\mathcal {F}(v)$ is continuous and f-invariant for $v\in \Omega $ . Moreover, if f is transitive and E is $C^\infty $ somewhere on a leaf then $\Omega $ is open and dense and the isomorphism class of any tangent cone is constant on $\Omega $ . The same statements hold for f replaced by a flow $\varphi _t$ .

In this setting, we call the isomorphism class of tangent cones obtained in Corollary 4.6 the Carnot nilpotent structure associated to for $\varphi _t$ .

Proof. The first claim follows from Theorem 4.2. The topological transitivity of f implies that there is a dense orbit in X which must intersect $\Omega $ since $\Omega $ is open. If E is $C^\infty $ on any open set of a leaf then $\Omega $ is also non-empty and therefore dense since it is invariant under f. The isomorphism class of tangent cones on $\Omega $ is constant because isomorphism classes are constant on a dense orbit, depend continuously on the underlying point, and the space of isomorphism classes is Hausdorff by Theorem 2.3. Lastly, the proof holds verbatim for flows by replacing f by $\varphi _t$ .

Kalinin [Kal11] defined a notion of closing property for diffeomorphisms. We define the analogous notion of closing property for flows.

Definition 4.7. A flow $\varphi _t:X\to X$ of a compact manifold X has the stable closing property (with exponent $\lambda $ ) if there are $c,\lambda , \delta _0>0$ with $\lambda>0$ such that for every $x\in X$ and some $T\in \mathbb {R}$ with $d(x,\varphi _T(x))<\delta _0$ there are $p,y\in X$ such that

1. (1) p is periodic with period $T_0$ where $|T_0-T|<\delta _0$ , and

2. (2) $d(\varphi _t(x),\varphi _t(p))<c e^{-\lambda \min \{t,T_0-t\}}d(x,\varphi _T(x))$ for $0\le t\le T_0$ , and

3. (3) $d(\varphi _t(p),\varphi _t(y))<c e^{-\lambda t}d(x,\varphi _T(x))$ and $d(\varphi _t(x),\varphi _t(y))<c e^{-\lambda (T_0-t)}d(x,\varphi _T(x))$ .

Here y plays a connecting role of being stably related to p and unstably related to x.

Examples of dynamical systems having the closing property include Anosov flows and diffeomorphisms and flows over subshifts of finite type (cf. [Kal11]).

We deduce Theorem 1.6 from the introduction as a conclusion of this section.

Theorem 1.6. Assume Hypotheses 4.1. Suppose the distribution E is uniformly $C^k$ for a sufficiently large k and $\mathcal {F}$ is transversally Hölder continuous. Also assume that the $C^1$ diffeomorphism f (or flow $\varphi _t$ ) is topologically transitive, satisfies the stable closing property and that is transversally Hölder continuous. Lastly, suppose that the graded nilpotent group associated to D on $\Omega $ is asymmetric and $(r+1)$ -step. Let $\mu $ be any finite f-invariant (respectively, $\varphi _t$ -invariant) ergodic measure whose support $\operatorname {Supp}(\mu )$ satisfies $\operatorname {Supp}(\mu )\cap \Omega \neq \emptyset $ . Then the Lyapunov exponents of f (respectively, $\phi _1$ ) along $\mathcal {F}$ are $\log \lambda , 2\log \lambda , \ldots , (r+1)\log \lambda $ with multiplicities $\dim E,\dim E^{1}-\dim E,\ldots ,\dim E^{r}-\dim E^{r-1}$ for some $\lambda =\lambda _\mu>1$ .

Proof. Recall that $\Omega $ is the subset obtained from Corollary 4.6. Let $v\in \Omega $ be a periodic point of period T. It follows that is a graded automorphism of the nilpotent group $TC_v\mathcal {F}(v)$ . On the other hand, the isomorphism class of $TC_v\mathcal {F}(v)$ is asymmetric. Hence is a homothety. Suppose $TC_v\mathcal {F}(v)$ is an $(r+1)$ -step Carnot nilpotent Lie group. By Theorem 1.4, there exists $\lambda (v)$ such that the Lyapunov exponents of $f^T$ along $\mathcal {F}$ for v are $T\log \lambda (v), 2T\log \lambda (v), \ldots , (r+1)T \log \lambda (v)$ .

Since $\operatorname {Supp}(\mu )\cap \Omega \neq \emptyset $ , $\Omega $ has full $\mu $ -measure as it is invariant and $\mu $ is ergodic. By [Kal11, Proposition 3.2], Lyapunov exponents of with respect to any invariant ergodic measure are approximated by those at periodic points. (This proposition is stated for standard $\operatorname {GL}(n,\mathbb {R})$ -valued cocycles; however, following the discussion after [GS97, Proposition 2.6] these estimates may be applied to derivative cocycles as well.) Therefore, there is $\lambda>1$ such that Lyapunov exponents of f along $\mathcal {F}$ with respect to $\mu $ are $\log \lambda , 2\log \lambda , \ldots , (r+1)\log \lambda $ . The proof in the case of flows follows verbatim after replacing f by $\varphi _t$ .

While we need greater regularity in Theorem 1.6, the Hölder continuity of E often follows from the dominated splitting assumption. For example, we have the following theorem (cf. also [BP07, Theorem 5.3.2]).

Theorem 4.8. [CP15, Theorem 4.11 and Remark 4.12]

Let $f: X \rightarrow X$ be a $C^{2}$ diffeomorphism and $\Lambda \subset X$ a compact f-invariant set admitting a dominated splitting of the form $T_{\Lambda } X=E \oplus F .$ Then there is $\theta \in (0,1]$ such that the bundles E and F are $\theta $ Hölder continuous. Moreover, $\theta \in (0,1]$ can be chosen to be any number such that, for all sufficiently large $N\ge 1$ ,

Remark 4.9. The analogous statement for flows $\varphi _t:X\to X$ follows from the above result applied to the time-one map. Moreover, the same proof works for arbitrary $C^{\alpha }$ cocycles over $C^1$ diffeomorphisms or flows admitting a dominated splitting for $\alpha \in (0,1]$ .

Lastly, note that Anosov flows with $E=E^s$ is a special case (cf. [Bal95, Appendix by Brin, Proposition 4.4]).

In the next sections we consider applications of our previous results to sundry geometric settings.

5 Application: local rigidity of smooth dominated splittings

In this section we will present some cases where the existence of a leafwise smooth invariant splitting of the unstable bundle leads to some form of rigidity. This provides some evidence for positive answers to Questions 1.8. However, we first show that smooth time changes of flows with uniformly $C^k$ slow distributions along the unstable foliation maintain this property. Hence, in general, the best rigidity we may expect is smooth orbit equivalence.

5.1 Time changes

Let $\varphi _t:{N}\to {N}$ be the $C^{\infty }$ flow generated by the vector field $x\mapsto X_x$ . Given a positive $C^\infty $ function $\ell :{N}\to (0,\infty )$ , consider the $C^\infty $ flow $\psi _t$ whose generating vector field is $x\mapsto \ell (x) X_x$ . The cumulative time change $\tau :{N}\times \mathbb {R}\to \mathbb {R}$ is given by $\tau (x,t)=\int _0^t \ell (\psi _s x)\,ds$ , and hence $\psi _t(x)=\varphi _{\tau (x,t)}(x)$ . The inverse function $\sigma (x,t)=\tau ^{-1}(x,t)$ satisfies $\varphi _t(x)=\psi _{\sigma (x,t)}(x)$ and corresponds to the cumulative time change for the field rescaling function $k(x)={1}/{\ell (x)}$ .

Anosov and Sinai [AS67] showed that if $\varphi _t$ is a $C^{\infty }$ Anosov flow and $\psi _t$ is obtained from $\varphi _t$ by the multiplication of speeds by $\ell $ , then $\psi _t$ is Anosov. In fact, they showed that we have a hyperbolic splitting

$$ \begin{align*} T{N}=E^{u}_{\psi_t}\oplus E^{s}_{\psi_t}\oplus E^0_{\psi_t} \end{align*} $$

for $D \psi _{t}$ where

$$ \begin{align*} E^{u}_{\psi_t}(x)=\{ v+z(x,v)X_x \,:\, v\in E^u_{\varphi_t}(x) \} \end{align*} $$

for some real-valued function $z(x,\cdot ):E^u_{\varphi _t}(x)\to \mathbb {R}$ which is linear in its second argument and continuous in x. Following Parry [Par86], we may express the function z as

$$ \begin{align*} z(x, \xi)=-\ell(x) \int_{0}^{\infty}(D \varphi_{s} \xi(\ell) / \ell(\varphi_{s} x)^{2})\, ds=\ell(x) \int_{0}^{\infty} \xi(k \circ \varphi_{s})\, ds. \end{align*} $$

We now observe that if $\ell $ is smooth then so are $\tau $ , k and $\sigma $ as functions on N or $N\times \mathbb {R}$ . Moreover, if the invariant splitting for $E^u_{\varphi _t}=E^u_{\mathrm {slow},\varphi _t}\oplus E^u_{\mathrm {fast},\varphi _t}$ is leafwise smooth, then the graphs

$$ \begin{align*} E^u_{\mathrm{slow},\psi_t}&=\{ v+ z(x,v)X_x \,:\, v\in E^u_{\mathrm{slow},\varphi_t} \}, \\ E^u_{\mathrm{fast},\psi_t}&=\{ v+ z(x,v)X_x \,:\, v\in E^u_{\mathrm{fast},\varphi_t} \} \end{align*} $$

remain leafwise smooth and invariant under ${\psi _t}$ . This last statement follows from the formula

$$ \begin{align*} D \psi_{\sigma(x,t)}(v+z(x, v) X_{x})=D \varphi_{t} v+z(\varphi_{t} x, D \varphi_{t} v) X_{\varphi_{t}(x)} \end{align*} $$

which we apply in forward time to $E^u_{\mathrm {slow},\psi _t}$ and in backward time in the case of $E^u_{\mathrm {fast},\psi _t}$ . From the explicit formula for $z(x,v)$ we see that its overall regularity restricted to the subbundle $E^u_{\mathrm {slow},\varphi _t}$ will be the same as that of $E^u_{\mathrm {slow},\varphi _t}$ since $\ell $ , k and $\varphi _s$ are smooth.

We summarize this discussion in the following proposition.

Proposition 5.1. If a $C^\infty $ Anosov flow $\varphi _t$ has a uniformly $C^k$ slow unstable distribution along its unstable foliation, then a smooth time change $\psi _t$ also has this property.

Observe that, in general, such $C^\infty $ time changes $\psi _t$ are not conjugate to $\varphi _t$ as periodic orbits may have different periods. To obtain explicit examples, we may apply the above proposition to geodesic flows of compact locally symmetric spaces or suspensions of Anosov diffeomorphisms with smooth slow distributions. Flows of the latter type arise from algebraic examples of splittings, for example from products or toral automorphisms.

5.2 Geodesic flows

We next apply the discussion on sub-Riemannian dynamics from the previous section to geodesic flows of closed manifolds of negative sectional curvature. We will prove a local rigidity statement for certain perturbations of a locally quaternionic hyperbolic or octonionic hyperbolic closed manifold.

Definition 5.2. Let $\mathcal {A}$ be a topological space and let $\{ \mathcal {F}_\rho \}_{\rho \in \mathcal {A}}$ be a parametrized family of foliations of a manifold M. We say that a parametrized family of dominated splittings $\{ T\mathcal {F}_\rho =E_\rho \oplus F_\rho \}_{\rho \in \mathcal {A}}$ is uniformly $C^k$ continuous in the parameter if $E_\rho $ and $F_\rho $ are $C^k$ along each leaf of the foliation $\mathcal {F}_{\rho }$ and their derivatives up to order k vary continuously in the parameter $\rho $ with respect to the given topology on the index set $\mathcal {A}$ . We say two splittings $E_\rho \oplus F_\rho $ and $E_{\rho _0}\oplus F_{\rho _0}$ from a uniformly $C^k$ family of splittings are $\epsilon $ -uniformly $C^k$ -close if their derivatives up to order k along their respective foliations $\mathcal {F}_{\rho }$ and $\mathcal {F}_{\rho _0}$ are $\epsilon $ -close.

Remark 5.3. By the permanence theorem [HPS77, Theorem 6.8] the $W^u_g$ foliations for the geodesic flow of g, though only transversely Hölder on all of M, have leafwise smooth distributions that vary continuously in the $C^k$ topology as g varies continuously in the $C^{k+1}$ topology. Even though $E^u_{\mathrm {fast},g}$ is uniformly $C^k$ along $W^u_g$ on each $(M,g)$ , $E^u_{\mathrm {slow},g}$ is only continuous along $W^u_g$ in general. However, even when $E^u_{\mathrm {slow},g}$ is assumed to be $C^\infty $ along leaves of $W^u_g$ , it is not necessarily uniformly $C^k$ on M and may not even be uniformly $C^k$ in the metric parameter g as it varies in the $C^{k+1}$ topology for any $k>0$ . We will need to assume the latter property in some of the statements below.

Consider a smooth n-manifold X. For $n_1\leq n_2\leq \dim X$ , let $\operatorname {Flag}(n_1,n_2)\to X$ be the smooth fiber bundle whose fiber $\operatorname {Flag}_x(n_1,n_2)$ is the space of all 2-flags of $n_1$ -dimensional subspaces of $n_2$ -dimensional subspaces of $T_xX$ . Now let $\mathscr {EF}$ be the subset of $C^0$ sections $x\to (E_x\subset F_x) \in \operatorname {Flag}_x(n_1,n_2)$ such that the corresponding distribution F generates a $C^0$ foliation $\mathcal {F}$ with $C^\infty $ leaves and the E distribution is $C^\infty $ along the leaves of the foliation $\mathcal {F}$ . We write these sections as $E\subset F$ for corresponding distributions E and F. The $C^{0,k}$ topology on $\mathscr {EF}$ is the maximal refinement of the subspace topology inherited from the compact-open topology on $C^0$ sections of $\operatorname {Flag}(n_1,n_2)$ such that the partial derivatives up to order k of E along the leaves of $\mathcal {F}$ vary continuously.

Lemma 5.4. Let X be a smooth closed manifold with a pair of distributions $(E_0\subset F_0) \in \mathscr {EF}$ such that $E_{0}$ restricted to each leaf is generic of order $r+1$ along the leaves of $\mathcal {F}$ . Then there is an open set $U\subset \mathscr {EF}$ in the $C^{0,r}$ topology such that each $(E\subset F)\in U$ is generic of order $r+1$ and the subbundles $E=E^0,E^1,\ldots ,E^r$ of iterated bracket spaces vary continuously as subbundles of $TX$ .

Proof. Since X is closed, $F_0$ defines a foliation $\mathcal {F}$ , and $E_0$ is a given distribution, we may find a finite open cover of foliation charts for $(X,\mathcal {F})$ where on each open chart V, the distribution $E_0$ is spanned by continuous vector fields $X_1,\ldots ,X_{d_0}$ which are $C^\infty $ on each leaf of $\mathcal {F}$ and whose successive brackets at each point x generate a filtration $E_0(x)=E^0_0(x)\subset E_0^1(x)\subset E_0^r(x)=F_x$ and fit together to form subbundles locally and hence globally.

Since the $X_i$ vary continuously and the iterated brackets along each leaf vary continuously in the parameters of the $C^{0,r}$ topology, there is an open neighborhood $U\subset \mathscr {EF}$ of $(E_0\subset F_0)$ in the $C^{0,r}$ topology for which the iterated brackets of the nearby $X_i$ spanning E continue to be non-zero on all of V. (For sufficiently close foliations $\mathcal {F}$ for the distribution F, the open set V still serves as a chart.)

Since there are only finitely many charts V, by intersecting the resulting open sets U we may find a common neighborhood.

Lemma 5.5. Let $(M,g_0)$ be a closed negatively curved manifold admitting a leafwise $C^\infty $ dominated splitting $E^u_{\phi _t^0}=E^u_{\mathrm {fast},\phi _t^0}\oplus E^u_{\mathrm {slow},\phi _t^0}$ for its geodesic flow $\phi _t^0$ . Assume that $(E^u_{\mathrm {slow},\phi _t^0}\subset E^u_{\phi _t^0})\in \mathscr {EF}$ and that $E^u_{\mathrm {slow},\phi _t^0}$ is generic and horizontal of order $r+1$ along the leaves of $W^u_{\phi _t^0}$ . Recall that by Proposition 2.2 there is a $C^{1}$ neighborhood U of $\phi _t^0$ such that any flow $(SM,\phi _t)\in U$ has a dominated splitting $E^u_{\mathrm {fast},\phi _t}\oplus E^u_{\mathrm {slow},\phi _t}$ . If we further assume that this splitting is $C^\infty $ along unstable leaves and sufficiently uniformly $C^r$ -close (along the unstable foliation) to that of $\phi _t^0$ then $(E^u_{\mathrm {slow},\phi _t}\subset E^u_{\phi _t})\in \mathscr {EF}$ and $E^u_{\mathrm {slow},\phi _t}$ is generic and horizontal of order $r+1$ along the leaves of $\mathcal {F}$ . Furthermore, the tangent cone map $v\mapsto TC_vW^u_{\phi _t}$ associated to $(M,\phi _t)$ exists and is $C^0$ -close to $v\mapsto TC_vW^u_{\phi _t^0}$ , the tangent cone map for $\phi _t^0$ , with respect to the Hausdorff topology on the moduli space $\mathcal {NL}_{n-1}/\mathrm {GL}(n-1,\mathbb {R})$ .

Proof. Recall that the dominated splitting condition is an open condition in the $C^r$ topology and thus it holds on some neighborhood U of $\phi _t^0$ .

As in the proof of Lemma 4.5, we exploit the fact that both the Carnot metric and the algebraic structure of the tangent cone depend on the construction of the Métivier fields $\widehat {X}_{j,k;x}$ .

By assumption, the 2-flag of distributions $E^u_{\mathrm {slow},\phi _t}\subset E^u$ are $C^\infty $ along $W^u_{\phi _t}$ and uniformly close to $E^u_{\mathrm {slow},\phi _t^0}\subset E^u_{\phi _t}$ in the $C^r$ topology. Provided they are close enough, then by Lemma 5.4 $E^u_{\mathrm {slow},\phi _t}$ is generic of order $r+1$ , with nearby subbundles of iterated bracket spaces. In particular, $E^u_{\mathrm {slow},\phi _t}$ is horizontal as well. Moreover, the construction of the $\widehat {X}_{r-i,j;x}$ for $i=0,\ldots ,r$ varies continuously in the $C^i$ topology on the vector fields $X_{0,1;x},\ldots , X_{0,d_0;x}$ locally spanning $E^u_{\mathrm {slow},\phi _t}$ . By hypothesis the latter fields depend continuously in the $C^r$ topology in $\phi _t$ . We conclude as in the proof of Lemma 4.5 and Proposition 3.4 that the nilpotent structure in $\mathcal {NL}_{n-1}/\mathrm {GL}(n-1,\mathbb {R})$ is $C^0$ -close as claimed.

Remark 5.6. We observe that if metrics vary in the $C^{r+1}$ topology, then the geodesic flows vary in the $C^r$ topology. Hence the same lemma holds for metrics except that U is a $C^2$ neighborhood of $g_0$ .

Theorem 1.7. Let $\phi _t^0$ be the geodesic flow on a locally quaternionic hyperbolic or octonionic hyperbolic closed manifold M. Then if $\phi _t$ is any $C^\infty $ flow $C^1$ -close to $\phi _t^0$ for which $E^u_{\mathrm {slow},\phi _t}$ remains $C^\infty $ along unstable leaves and is sufficiently uniformly $C^1$ -close (in the sense of Definition 5.2) to that of $\phi _t^0$ , then $\phi _t$ is $C^\infty $ orbit equivalent to $\phi _t^0$ .

Proof. We observe that the symmetric flow $\phi _t^0$ has a dominated splitting $E^u=E^u_{\mathrm {fast}}\oplus E^u_{\mathrm {slow}}$ which is globally $C^\infty $ and for which $E^u_{\mathrm {slow}}$ is generic of order one and horizontal. By Lemma 5.5 there is a $C^1$ open neighborhood U of $\phi _t^0$ where $E^u_{\mathrm {slow},\phi _t}$ is generic of order one and horizontal whenever $\phi _t\in U$ has a dominated splitting which is $C^\infty $ along unstable leaves and uniformly $C^1$ -close along the unstable foliation to that of $\phi _t^0$ . Moreover, the associated tangent cones $TC_vW^u(v)$ for $\phi _t$ are $C^0$ -close to those of $\phi _t^0$ .

Recall that by Examples 2.8 the tangent cone associated to $(M,\phi _t^0)$ is asymmetric. By Corollary 2.10, after possibly shrinking U, the tangent cone associated to $(M,\phi _t)$ is also asymmetric. We can shrink U to an even smaller set so that the tangent cone associated to $(M,\phi _t)$ is 2-step. Thus by Theorem 1.6, the unstable Lyapunov exponents are $\log \lambda $ and $2\log \lambda $ for a $\lambda>1$ . The multiplicity of the exponents are the dimensions of the slow and the fast unstable distribution. Therefore, the Lyapunov spectra of the geodesic flows on $(M,\phi _t)$ and $(M,\phi _t^0)$ are proportional.

We wish to apply [But19, Theorem 3.2] which is stated for the so-called horizontal measure, $\mu _M$ . We need to verify that this measure has full support. We observe that by [But19, §2.8] this is a Gibbs state for a Hölder potential. These have full support in our setting ([PPS15, p. 50] or [CPZ19, Theorem 4.11]).

Remark 5.7. By the Livsic theorem, whenever two orbit equivalent flows have the same periods on periodic points they are conjugate. The regularity of the conjugacy will be at least one less than the regularity of the time change.

Theorem 1.9. Let $g_0$ be a locally quaternionic hyperbolic or octonionic hyperbolic metric on a smooth closed manifold M. Then $g_0$ is locally rigid within the family of $C^2$ -close $C^\infty $ metrics whose $E^u_{\mathrm {slow}}$ remains $C^\infty $ along unstable leaves and is sufficiently uniformly $C^1$ -close (in the sense of Definition 5.2) to that of $g_0$ .

Remark 5.8. The above theorem is reminiscent of the global rigidity result (Theorem 3) in [BFL92]. While we do not assume any transverse regularity of the splitting in our theorem, we do require that the metric is nearby the locally symmetric one. We also note that there is a $C^2$ open neighborhood of $g_0$ such that the unstable distributions of the perturbed manifolds admit dominated splittings by Proposition 2.2.

Proof. The proof is nearly identical to that of Theorem 1.7, except that we note by Remark 5.6 there is a $C^2$ open neighborhood U of $g_0$ with the desired properties. Moreover, we finish using Theorem 1.5 of [But19] instead of Theorem 3.2.

Remark 5.9. If $g_0$ is any metric as in Lemma 5.5 whose associated nilpotent tangent cone has an asymmetric Lie algebra, then by Theorem 1.6 its unstable Lyapunov spectrum is of the form $\log \lambda ,2\log \lambda , \ldots , (r+1) \log \lambda $ for some $\lambda>1$ . We can conclude by the proof of Theorem 1.9 that any metric that is sufficiently uniformly $C^2$ -close along unstable leaves will also have a Lyapunov spectrum of the same form.

5.3 Diffeomorphisms of nilmanifolds

In this section we apply our results to perturbations of automorphisms of certain nilmanifolds with smooth dominated splittings.

Standing assumption: Let M be a closed manifold and let $f_0$ be a transitive $C^\infty $ Anosov diffeomorphism such that tangent cones of unstable leaves exist everywhere and are isomorphic to a fixed asymmetric $(r+1)$ -step Carnot nilpotent Lie group N. Let $E^u_{\mathrm {slow},f_0}$ be the horizontal distribution along unstable leaves that gives rise to the tangent cone structure.

We obtain the following local rigidity theorem.

Theorem 1.10. There is a $C^1$ open neighborhood U of $f_0$ in $\mathrm {Diff}{}^\infty (M)$ such that if $f \in U$ admits a smooth splitting $E^u_f=E^u_{\mathrm {fast},f}\oplus E^u_{\mathrm {slow},f}$ along unstable leaves with $\dim (E^u_{\mathrm {slow},f})=\dim (E^u_{\mathrm {slow},f_0})$ , and $E^u_{\mathrm {slow},f}$ is sufficiently uniformly $C^{r}$ -close along unstable leaves to $E^u_{\mathrm {slow},f_0}$ , then for any invariant ergodic measure $\mu $ there is $\lambda _\mu>1$ such that the unstable Lyapunov exponents of f with respect to $\mu $ , are $\log \lambda _\mu ,2\log \lambda _\mu ,\ldots ,(r+1) \log \lambda _\mu $ with the same multiplicity as for $f_0$ .

Proof. By the construction of $f_0$ , tangent cones of unstable manifolds of $f_0$ exist everywhere and are isomorphic as graded nilpotent Lie groups to N.

We argue as in the proof of Theorem 1.9. The tangent cones of unstable manifolds of f exist everywhere, and such tangent cones are close to N in the variety of Carnot nilpotent Lie groups. By Corollary 2.10, those tangent cones are asymmetric. By Theorem 1.6, there is $\lambda _\mu>1$ such that all unstable Lyapunov exponents, without multiplicity, of f with respect to $\mu $ , are $\log \lambda _\mu ,2\log \lambda _\mu ,\ldots ,(r+1)\log \lambda _\mu $ . The multiplicities are given by the dimensions of the bracket spaces starting with $E^u_{\mathrm {slow},f}$ which have the same dimensions as for $f_0$ .

Recall that for any transitive Anosov diffeomorphism f there is a unique invariant Sinai–Ruelle–Bowen (SRB) measure, which we call $\sigma $ (see [KH95, 20.3.8] and [Sin72]). For $f_0$ as in Theorem 1.10, we denote by $\sigma _0$ the unique SRB measure for $f_0$ . In the next corollary we obtain a rigidity statement under an additional assumption which is analogous to a higher hyperbolic rank condition (cf. below).

Corollary 5.10. Suppose we are in the setting as Theorem 1.10 with M a nilmanifold and $f_0$ an automorphism. In addition, assume $\lambda _{\sigma }(f)\geq \lambda _{\sigma _0}(f_0)$ . Then $\sigma $ is the measure of maximal entropy for f and has the same Lyapunov spectrum as $\sigma _0$ for $f_0$ .

Proof. Set $\dim E^{-1}=0$ and let $d_i=\dim E^{i}-\dim E^{i-1}$ for $i=0,\ldots ,r$ . By the Pesin formula and Theorem 1.10, we also have $h_{\sigma }(f)=\sum _{i=0}^r (i+1) \log \lambda _{\sigma }(f) d_i$ . Moreover, by structural stability f and $f_0$ are conjugate and so they share the same topological entropies $h_{\mathrm {top}}(f)=h_{\mathrm {top}}(f_0)$ . Since $f_0$ is algebraic, the Lyapunov exponents are independent of the measure and so Haar measure $\sigma _0$ is the measure of maximal entropy. Together with our assumptions, we have

$$ \begin{align*} h_{\mathrm{top}}(f)&\geq h_{\sigma}(f)=\sum_i i \log\lambda_{\sigma}(f) d_i\\[-4pt] &\geq \sum_i i \log\lambda_{\sigma_0}(f_0) d_i =h_{\sigma_0}(f_0)=h_{\mathrm{top}}(f_0)=h_{\mathrm{top}}(f). \end{align*} $$

Hence all of the inequalities are equalities. Therefore $h_{\mathrm {top}}(f)=h_{\sigma }(f)$ and hence $\sigma $ is the measure of maximal entropy for f. Moreover, $\lambda _{\sigma }(f)=\lambda _{\sigma _0}(f_0)$ and so the spectra coincide.

As an application we describe a class of examples of special Anosov nil-automorphisms.

Definition 5.11. A nilpotent Lie algebra $\mathfrak {n}$ is called rational if it is a Lie algebra over $\mathbb {Q}$ . A nilpotent Lie group N is called rational if there is a rational Lie algebra $\mathfrak {n}$ such that $Lie(N)=\mathfrak {n} \otimes \mathbb {R}$ .

Example 5.12. [Lau03, Theorem 3.1] Let N be a graded rational nilpotent group. Then there is a lattice $\Gamma <N\times N$ such that the nilmanifold $M=(N\times N)/\Gamma $ admits an Anosov automorphism in which unstable and stable manifolds are vertical and horizontal copies of N.

We obtain the following as an immediate corollary to Theorem 1.10.

Corollary 1.11. Let N be an asymmetric k-step Carnot rational nilpotent group. Let $f_0$ be the Anosov automorphism of $M=(N\times N)/\Gamma $ obtained from Example 5.12. Then there is a $C^1$ open neighborhood U of $f_0$ in $\mathrm {Diff}{}^\infty (M)$ such that if $f \in U$ admits a smooth dominated splitting $E^u_f=E^u_{\mathrm {fast},f}\oplus E^u_{\mathrm {slow},f}$ along unstable leaves and $E^u_{\mathrm {slow},f}$ is sufficiently uniformly $C^{r}$ -close along unstable leaves to $E^u_{\mathrm {slow},f_0}$ , then for any invariant ergodic measure $\mu $ there is $\lambda _\mu>1$ such that the unstable Lyapunov exponents of f with respect to $\mu $ are $\log \lambda _\mu ,2\log \lambda _\mu ,\ldots ,(r+1)\log \lambda _\mu $ with the corresponding multiplicities as for $f_0$ .

Remark 5.13. In the above corollary, one may replace the hypothesis of a smooth dominated splitting with that of a slow distribution $E^u_{\mathrm {slow},f}$ smooth along unstable leaves. This readily follows from Proposition 2.2.

Even though the Heisenberg group is not asymmetric, we may still obtain resonance of Lyapunov exponents for Anosov diffeomorphisms on products of this group.

Example 5.14. The following example due to Borel [Sma67, Example 1 in I-(3.8)] provides an Anosov diffeomorphism on a nilmanifold whose unstable tangent cones are not asymmetric nilpotent groups. Let N be the three-dimensional Heisenberg group. Consider $(N\times N)/\Gamma $ where $\Gamma $ is the lattice subgroup whose elements (with respect to the standard basis on the Lie algebra) are of the form $(\alpha ,\beta ,\gamma ,\alpha ^{\sigma },\beta ^{\sigma },\gamma ^{\sigma })$ for $\alpha ,\beta ,\gamma \in \mathbb {Z}[\sqrt {3}]$ and where $\sigma $ represents the Galois automorphism with $(a+b\sqrt {3})^\sigma =a-b\sqrt {3}$ . Let $f_0$ be the Anosov diffeomorphism induced from the Lie algebra map $A(a,b,c,d,e,f)=(\lambda a,\lambda ^2 b,\lambda ^3 c,\lambda ^{-1} d,\lambda ^{-2} e,\lambda ^{-3}f)$ for $\lambda =2+\sqrt {3}$ . Here the unstable distribution corresponds to the first factor of N with coordinates $(a,b,c)$ and the slow distribution has coordinates $(a,b)$ which is the standard horizontal (and generic) distribution of N.

We may obtain a similar corollary to Corollary 1.11 for the above example.

Corollary 1.12. Let N be the three-dimensional Heisenberg group. Let $\Gamma $ be the lattice and $f_0$ the Anosov automorphism of $M=(N\times N)/\Gamma $ obtained from Example 5.14. Then there is a $C^1$ open neighborhood U of $f_0$ in $\mathrm {Diff}{}^\infty (M)$ such that if $f \in U$ admits a smooth splitting $E^u_f=E^u_{\mathrm {fast},f}\oplus E^u_{\mathrm {slow},f}$ along unstable leaves and $E^u_{\mathrm {slow},f}$ is sufficiently uniformly $C^{1}$ -close along unstable leaves to $E^u_{\mathrm {slow},f_0}$ , then for any invariant ergodic measure $\mu $ the unstable Lyapunov exponents for f are $\log \lambda _1,\log \lambda _2$ and $\log \lambda _1+\log \lambda _2$ for some $\lambda _1>1$ and $\lambda _2>1$ depending on $\mu $ .

Proof. We note that this result follows immediately from the analogue of Theorem 1.10 for Heisenberg groups in place of asymmetric groups and resonance in place of arithmeticity in the conclusion. The proof may be duplicated except for two points.

First, we need to show that, under the perturbation, the unstable tangent cones for f are still isomorphic to the Heisenberg group. Since $E^u_{\mathrm {slow},f}$ is sufficiently uniformly $C^1$ -close along unstable leaves to $E^u_{\mathrm {slow},f_0}$ , the horizontal distribution of $TC_vW^u_f(v)$ remains codimension-one, horizontal and generic. Hence $TC_vW^u_f(v)$ is a 2-step Carnot nilpotent Lie group with one-dimensional center for every $v\in SM$ . Moreover, the tangent cones for f are $C^0$ -close to those for $f_0$ . The Lie bracket induces a bilinear form $[\cdot ,\cdot ]:\mathfrak {n}_v^0\times \mathfrak {n}_v^0\to \mathfrak {n}_v^1$ between the first and second levels $\mathfrak {n}_v^0$ and $\mathfrak {n}_v^1$ of the Lie algebra $\mathfrak {n}_v$ of $TC_vW^u_f(v)$ , and it remains non-degenerate. Consequently, as the center is one-dimensional and there is only one non-degenerate skew symmetric bilinear form on $\mathfrak {n}_v^0$ for every $v\in SM$ , $TC_vW^u(v)$ is isomorphic as a graded nilpotent group to the Heisenberg group N.

Second, we need an analogous result to Theorem 1.6. In turn, we observe that in the proof of Theorem 1.6, we may replace Theorem 1.4 with Theorem 1.3 to obtain the corresponding analogue for the Heisenberg group in the hypotheses and the resonance in the conclusion. Having done so, the analogous result follows.

It is easy to construct examples of Anosov diffeomorphisms with integrable slow distributions on tori or products of nilmanifolds. One might ask whether for irreducible Anosov automorphisms of non-abelian nilpotent groups the slow distribution must be horizontal and generic. However, this is not the case. Indeed, the slow unstable distribution of [Shu69, Example 2 on p. 189] is integrable.

As our final application of smooth splittings we will investigate the local rigidity of the projective action of a quaternionic or octonionic hyperbolic lattice on its ideal boundary sphere.

5.4 Local rigidity of projective boundary actions

Sullivan [Sul85] initiated the study of local rigidity of lattice actions on the boundary. Ghys [Ghy93] introduced a suspension construction which relates actions of fundamental groups of surfaces on the circle to Anosov flows. This was later adapted by Yue [Yue95] to the study of lattice actions on higher-dimensional spheres. We will employ this construction to obtain a rigidity theorem for boundary actions (cf. also [Asa17, Theorem 6.1]).

Consider a rank-one symmetric space X with connected component $G<\mathrm {Isom}(X)$ of its isometry group. As is well known, the ideal boundary $\partial X$ of X may be identified as $\partial X=G/P$ for a (minimal) parabolic subgroup P. For a discrete subgroup $\Gamma < \mathrm {Isom}(X)$ , let $\rho _0$ be the action of $\Gamma $ on $\partial X$ induced from the action of $\Gamma $ on X. Note that $\rho _0$ is nothing but the left action on $G/P$ and thus is $C^\infty $ . This action preserves the projection to the boundary of the slow unstable distribution $E_0$ for the geodesic flow since the stable foliation and the holonomy along it are $C^\infty $ and $\Gamma $ -equivariant.

We briefly recall the suspension construction mentioned above. Let $\rho :\Gamma \to \operatorname {Diff}{}^\infty (\partial X)$ be a $C^1$ -close perturbation of $\rho _0$ . More precisely, we assume that $\rho $ is $C^1$ -close to $\rho _0$ on a fixed finite set of generators of $\Gamma $ (we recall that lattices in semisimple groups are always finitely generated). Consider the unit tangent bundle $SM$ for the locally symmetric space and note $\widetilde M=X$ . We have a diffeomorphism between $SX$ and $(\partial X\times \partial X \backslash D)\times \mathbb {R}$ where D is the diagonal of $\partial X\times \partial X$ .

We also have a diffeomorphism $\widehat {q}:SX\to X\times \partial X$ via $v\mapsto \widehat {q}(v)=(\pi (v),v(\infty ))$ , where $\pi (v)$ is the projection of v to its base point. The map $\widehat {q}$ descends to a diffeomorphism $q:SM\to (X\times \partial X)/\widehat {\rho }_0$ where $\widehat {\rho }_0(\gamma )(x,\xi )=(\gamma x,\rho _0(\gamma ) \xi )$ for all $\gamma \in \Gamma $ and $(x,\xi )\in X\times \partial X$ . (Here $\Gamma =\pi _1(M)$ acts by deck transformations on X.)

As $\rho $ is $C^1$ -close to $\rho _0$ , there is a $C^\infty $ diffeomorphism ${f{{\kern-1pt}:{\kern-1pt}}(X\times \partial X)/\widehat {\rho }\to (X\times \partial X)/\widehat {\rho }_0}$ where $\widehat {\rho }(\gamma )(x,\xi )=(\gamma x,\rho (\gamma )\xi )$ by Proposition A.1. Observe that the actions of $\widehat {\rho }$ and $\widehat {\rho }_0$ both preserve the leaves of the first-factor foliation $\{ X\times \{ \xi \}:\xi \in \partial X \}$ . Moreover, the map f maps horizontal leaves in $(X\times \partial X)/\widehat {\rho }$ to leaves that are $C^1$ -close to horizontal leaves in $(X\times \partial X)/\widehat {\rho }_0$ . The manifolds $\widehat {q}^{-1}(X\times \{ \xi \})$ , as $\xi $ varies, are the leaves of the center-stable foliation $W^{cs}$ for the geodesic flow $\phi _t$ . For each $x\in X$ we let $\sigma _x: \partial X\to \partial X$ be the map which takes $\xi \in \partial X$ to the opposite endpoint of the geodesic line through x and $\xi $ . Then $\widehat {q}^{-1}(\{ (y,\sigma _y\circ \sigma _x(\xi )): y\in X \})$ is precisely the center-unstable manifold $W^{cu}(\widehat {q}^{-1}(x,\xi ))\subset SX$ for the geodesic flow $\phi _t$ on $S\widetilde M$ .

Let $V^{cs}$ be the $C^\infty $ foliation of $SM$ whose leaves for $v\in SM$ are the manifolds $V^{cs}(v)=q^{-1}\circ f([X\times {v(\infty )}])$ , where square brackets denote the $\widehat {\rho }$ -equivalence class. Note similarly that the center-stable foliation $W^{cs}$ on $SM$ for the locally symmetric metric has leaves $W^{cs}(v)=q^{-1}([X\times {v(\infty )}])$ where square brackets denote the $\widehat {\rho }_0$ -equivalence class.

Intersecting $V^{cs}\cap W^{cu}$ gives a $C^\infty $ one-dimensional foliation $\mathscr {F}$ which is $C^1$ -close to the locally symmetric geodesic foliation. Note that for all y in the geodesic line through x and $\xi $ , $\sigma _y\circ \sigma _x(\xi )=\xi $ . Hence the foliation $\mathscr {F}$ is the image under $q^{-1}\circ f$ of the quotient by $\widehat {\rho }$ of the symmetric geodesic foliation (of the first factor) in $X\times \partial X$ .

Using the locally symmetric metric, we obtain a norm on the tangent space to $\mathscr {F}$ from which we may construct a unit vector field Y on $SM$ tangent to these curves. By structural stability, the field Y defines a smooth Anosov flow $\psi _t$ which is Hölder orbit equivalent to the locally symmetric flow $\phi _t$ . (Note that by invariance of $V^{cs}$ and $W^{cu}$ , and the fact that $\psi _t$ is $C^1$ -close to $\phi _t$ , it follows that $V^{cs}$ and $W^{cu}$ are the center-stable and center-unstable foliations for $\psi _t$ .)

Theorem 1.16. Let X be a quaternionic hyperbolic space or the Cayley plane, and let $M=X/\Gamma $ for a cocompact lattice $\Gamma <\mathrm {Isom}(X)$ . Let $\rho :\Gamma \to \operatorname {Diff}{}^\infty (\partial X)$ be a $C^1$ -close perturbation of $\rho _0$ which still preserves a $C^\infty $ distribution E nearby to $E_0$ . Then $\rho $ is $C^\infty $ conjugate to $\rho _0$ .

Proof. Consider the slow distribution $E^u_{\mathrm {slow}}$ for $\psi _t$ . Since $V^{cs}$ is $C^\infty $ , the center-stable holonomy is $C^\infty $ . Moreover, $E^u_{\mathrm {slow}}\subset V^u$ is holonomy invariant (e.g. by [FK90, Lemma 4]; cf. [CNS20, Lemma 5.5]). Since $\phi _t$ has a dominated splitting on $W^{u}$ and is $C^1$ -close to $\psi _t$ , the latter flow also admits a dominated splitting of $V^{u}\subset W^{cu}$ and $E^u_{\mathrm {slow}}\subset V^{u}$ is also continuous. In what follows let $\psi _t^{\rho _0}$ denote the flow on $(X\times \partial X)/\widehat {\rho _0}$ given by pulling back $\psi _t$ under $q^{-1}$ . Similarly, let $\psi _t^{\rho }$ denote the flow on $(X\times \partial X)/\widehat {\rho }$ given by pulling back $\psi _t^{\rho _0}$ under f.

It is easier to understand the center-stable holonomy using $\psi _t^{\rho }$ instead of $\psi _t$ . The center-stable leaves on $(X\times \partial X)/\widehat {\rho }$ are quotients of leaves of form $X\times \{b\}$ , for $b\in \partial X$ . The center-stable holonomy on the cover $X\times \partial X$ of $(X\times \partial X)/\widehat {\rho }$ is the map preserving the second component in $\partial X$ . There is also a smooth map from each stable leaf in $X\times \partial X$ into $\partial X$ by projecting to the second factor. This map is invariant under center-stable holonomy and $\psi _t^{\rho }$ . It follows that $df^{-1}\circ q(E^u_{\mathrm {slow}})$ projects to a well-defined and smooth distribution on $\partial X$ . Consequently, we get a distribution on $\partial X$ that is $C^1$ -close to E and invariant under $\rho (\Gamma )$ . Inverses of projections from the boundary $\partial X$ onto unstable leaves maps E to a subdistribution of $E^u$ that is $C^1$ -close to $df^{-1}\circ q(E^u_{\mathrm {slow}})$ and is invariant under the flow $\psi _t^{\rho }$ . Thus these two distributions coincide. It follows that $df^{-1}\circ q(E^u_{\mathrm {slow}})$ projects to E under the projection onto the second factor $\partial X$ . Since E is $C^\infty $ and is $C^1$ -close to $E_0$ , we have that $E^u_{\mathrm {slow}}$ is horizontal and generic everywhere. By Theorem 3.2, we therefore have a tangent cone at each point of the leaves of $V^{cs}$ and this is a perturbation of the quaternionic or octonionic Heisenberg group by Corollary 4.6. The corresponding nilpotent group is asymmetric by Corollary 2.10. Finally, by Proposition 3.12, the Lyapunov exponents all have ratio two.

By [But19, Theorem 3.6], the flow $\psi _t$ is $C^\infty $ orbit equivalent to the flow $\phi _t$ by a diffeomorphism $G:SM\to SM$ . We lift G to the universal covers and conjugate by $\widehat {q}$ to obtain a diffeomorphism $\widehat {G}:X\times \partial X \to X\times \partial X$ which intertwines $(q^{-1})^*\phi _t$ and $\psi _t^\rho $ as well as their corresponding center-stable foliations whose leaves are of the form $X\times \{b\}$ for $b\in \partial X$ . Thus $\widehat {G}$ induces a well-defined diffeomorphism from $\partial X$ to $\partial X$ that intertwines the $\rho _0$ and $\rho $ actions. In other words, the perturbed $\Gamma $ -action is smoothly conjugate to the original $\Gamma $ -action.

6 Local hyperbolic rank rigidity

In this section we prove the local rigidity results of Theorems 1.14–1.16. We first show in §6.1 that perturbations of locally symmetric metrics with higher hyperbolic rank have the same hyperbolic rank as the locally symmetric metric. This is a key step for Theorem 1.14, whose proof we provide in §6.2.

We begin by recalling some facts from [CNS20] about hyperbolic rank.

Lemma 6.1. Suppose M is a closed Riemannian manifold of hyperbolic rank k and with sectional curvatures $0>\kappa \ge -1$ . Let $\widehat {\mathcal {E}}(v)\subset v^\perp $ be the subspace consisting of initial vectors w of unstable Jacobi fields of the form $t\mapsto e^t\|_{\varphi ^t v}w$ .

1. (1) $\widehat {\mathcal {E}}(v)$ is of dimension at least k for every v, by definition.

2. (2) The set $\widehat {\mathcal {R}}\subset SM$ where $\widehat {\mathcal {E}}(v)$ is of constant dimension k is open and dense by [CNS20, Lemma 2.4].

3. (3) The map $v\mapsto \widehat {\mathcal {E}}(v)$ is $C^\infty $ on $\widehat {\mathcal {R}}$ by [CNS20, Proposition 3.3].

4. (4) The lift $E_1^u(v)$ of $\widehat {\mathcal {E}}(v)$ to $ T_vW^u(v)=E^u(v)\subset TSM$ is a distribution on $\widehat {\mathcal {R}}$ which is uniformly $C^\infty $ along $W^u$ restricted to $\widehat {\mathcal {R}}$ which consists precisely of the Lyapunov spaces in $E^u$ of exponent $1$ , by [CNS20, Lemma 4.2]. (We will sometimes refer to the $E^u_1(v)$ space as $E^u_{\mathrm {fast}}(v)$ .)

5. (5) Denote by $\widehat {\mathcal {E}}(v)^\perp $ the orthogonal complement of $\widehat {\mathcal {E}}(v)$ within $v^\perp $ with respect to the Sasaki metric. The lift $E_{<1}^u$ of $\widehat {\mathcal {E}}^\perp $ to $E^u$ is the direct sum of unstable Lyapunov spaces of exponents different from $1$ , by [CNS20, Lemma 4.3]. (We will sometimes refer to the $E^u_{<1}(v)$ space as $E^u_{\mathrm {slow}}(v)$ .)

We remark that $E_{<1}^u$ is uniformly $C^\infty $ along $W^u$ restricted to $\widehat {\mathcal {R}}$ since it is orthogonal to $E^u_1$ in $E^u$ .

6.1 Holonomy groups of perturbations and hyperbolic rank

The goal of this subsection is to prove the following proposition and its Corollary 6.10.

Proposition 6.2. Let $(M, g_0)$ be a closed complex, quaternionic or octonionic hyperbolic locally symmetric manifold. Then there is an open $C^2$ neighborhood U of $g_0$ such that, for any $g\in U$ , if $(M,g)$ has higher hyperbolic rank and $\kappa _g\ge -1$ then $(M,g)$ has hyperbolic rank at least $1$ , $3$ or $7$ , respectively.

The proof of this proposition will rely on an analysis of the Brin–Pesin asymptotic holonomy group. Before giving the proof we will establish some notation and several lemmas.

Let $FM\to SM$ denote the full orthonormal frame bundle which is a (right) principal $\mathrm {SO}(n-1)$ bundle over $SM$ . Let $F_2M$ denote the $2$ -frame bundle over $SM$ which is a fiberwise quotient of $FM$ by $\mathrm {SO}(n-2)$ acting on the right, that is, we have bundle quotient maps

We endow $FM$ and $F_2M$ with the natural extensions of the Liouville measure by the Haar measures on the fiber. These measures are invariant under their respective frame flows. The k-frame flows commute with the corresponding right action of $\mathrm {SO}(n-1)$ on $F_kM$ . Consider the ergodic component $\mathcal {E}(v,w)$ of the 2-frame flow containing $(v,w)$ with respect to this measure. Moreover, assume $(v,w)$ is a generic 2-frame in the ergodic component $\mathcal {E}(v,w)$ . (This means that time averages over the orbit of $(v,w)$ coincide with the space average over $\mathcal {E}(v,w)$ with respect to its ergodic measure.) Without loss of generality we may assume $(v,w)$ is the first two vectors of a full frame f which is also generic in its ergodic component $\mathcal {E}(f)\subset FM$ for the full frame flow. Lemmas 5.1 (see the formulation of the ergodic component $Q(\omega )$ in the proof) and 5.2 of Brin [Bri82] show that

$$ \begin{align*} \mathcal{E}(f)=\overline{\bigcup_{t\in\mathbb{R}}\Phi_t(C(f))} \end{align*} $$

where $C(f)$ is the smallest set saturated by entire leaves of $W^s_f,W^u_f$ and containing f. Moreover, there is a compact subgroup $B^\infty _f< \mathrm {SO}(n-1)$ , called the ergodic component group, satisfying

$$ \begin{align*} \mathcal{E}(f)\cap \pi^{-1}_{n\to 1}(v)=f\cdot B^\infty_f. \end{align*} $$

This immediately implies that

$$ \begin{align*} \mathcal{E}(v,w)\cap \pi^{-1}_{2\to 1}(v)&=\pi_{n\to 2}(\mathcal{E}(f))\cap \pi^{-1}_{2\to 1}(v)\\&=\pi_{n\to 2}(f\cdot B^\infty_f)\cap \pi^{-1}_{2\to 1}(v)=f\cdot B^\infty_f \mathrm{SO}(n-2). \end{align*} $$

It is clear that if $f'=f\cdot h$ for $h\in \mathrm {SO}(n-1)$ then $B^\infty _{f'} = h^{-1}B^\infty _fh$ . And by definition if f and $f'$ are in the same ergodic component, then $B^\infty _f=B^\infty _{f'}$ . We call $B^\infty _f$ the (Brin–Pesin) asymptotic holonomy group, and denote its conjugacy class in $\mathrm {SO}(n-1)$ by $B^\infty $ .

Suppose $v\in SM$ and $v'\in W^s_{\phi _t}(v)$ . Fix $k\in \{ 2,\ldots ,n \}$ and let $p(v, v^{\prime })$ be the map from the fiber of the k-frame bundle, $F_{k} M$ , over v to the fiber over $v^{\prime }$ that takes each frame f to $p(v, v^{\prime })(f)=\pi ^{-1}_{k\to 1}(v^{\prime }) \cap W_{\Phi _t}^{s}(f).$ The map $p(v, v^{\prime })$ corresponds to a unique isometry between $v^{\perp }$ and $v^{\prime \perp }$ which commutes with the right action of $\mathrm {SO}(n-1)$ . We will mainly consider the case of the map for $k=2$ . After lifting to the universal cover, one can think of $p(v, v^{\prime })(f)$ as the result of ‘parallel transporting’ f along $\gamma _{v}(t)$ out to the boundary at infinity of $\tilde {M}$ and then back to $v^{\prime }$ along $\gamma _{v^{\prime }}(t)$ .

Similarly, when $v^{\prime }$ and v belong to the same leaf of $W_{\phi _t}^{u}$ , there is a map $p(v,v')(f)=\pi ^{-1}_{k\to 1}(v^{\prime }) \cap W_{\Phi _t}^{u}(f)$ . Following Brin (see [Bri82, Definition 4.4]) we define the transitivity group at v as follows.

Definition 6.3. Given any sequence $\sigma =\{v_{0}, v_{1}, \ldots , v_{k}\}$ with $v_{0}=v, v_{k}=\phi _{T}(v)$ for some $T\in \mathbb {R}$ such that each pair $\{v_{i}, v_{i+1}\}$ lies on the same leaf of $W_{\phi _t}^{s}$ or $W_{\phi _t}^{u}$ , we have an isometry of $v^{\perp }$ given by

$$ \begin{align*} I(\sigma,T)=\Phi_{-T} \circ \prod_{i=0}^{r} p(v_{i}, v_{i+1}). \end{align*} $$

The closure of the group generated by all such isometries with $v_0=v$ is denoted by $H_{v}<\mathrm {Isom}(v^\perp )$ and is called the transitivity group.

After picking an orthonormal full frame f with v as its first vector, $H_{v}$ can be identified as a subgroup of $\mathrm {SO}(n-1)$ . This identification depends on the choice of frame f. Different choices give conjugate identifications. It is well known that the ergodic component group $B^\infty _f<\mathrm {SO}(n-1)$ at a frame f over v coincides with the transitivity group $H_v$ under this identification (see the remark before Lemma 5.2, or Remark 2 of [Bri75a] and Proposition 2 of [Bri75b]). For clarity, we provide the details of this equivalence below.

Lemma 6.4. For each $k\in \{ 2,\ldots ,n \}$ , the ergodic component group coincides with the transitivity group at each $v\in SM$ and $f\in \pi _{k\to 1}^{-1}(v)$ , that is, $B^\infty _f=H_v$ , using the identification provided by the frame f.

Proof. Fix $v\in SM$ , and consider any continuous function $\alpha $ on the homogeneous space $\pi _{k\to 1}^{-1}(v)/H_v$ . Lift $\alpha $ to a function $\overline {\alpha }:\pi _{k\to 1}^{-1}(v)\to \mathbb {R}$ . Using the maps $p(v,v')$ for $v\in SM$ , we obtain a continuous extension $\widehat {\alpha }: F_kM\to \mathbb {R}$ given by $\widehat {\alpha }(f')=\overline {\alpha }(p(v',v)(f'))$ for any $f'\in \pi _{k\to 1}^{-1}(v')$ . Observe that $\widehat {\alpha }$ is well defined and invariant under both the action of $H_v$ and the frame flow. Since continuous functions on $\pi _{k\to 1}^{-1}(v)/H_v$ separate points, and continuous flow invariant functions on $F_kM$ are constant on ergodic components, we obtain a surjection from the space of ergodic components to $\pi _{k\to 1}^{-1}(v)/H_v$ . Therefore, $B^\infty _f< H_v$ for any $f\in \pi _{k\to 1}^{-1}(v)$ . On the other hand, as explained in [Bri75a, Remark 2], we also have $H_v< B^{\infty }_f$ .

Remark 6.5. Note that the entire formulation of the ergodic component groups, asymptotic holonomy groups and transitivity groups makes sense in the broader context of compact (isometric) group extensions of Anosov flows. Moreover, the above lemma also holds (see [Bri75b]).

For the next lemma, recall that an $\epsilon $ -net $F\subset X$ of a metric space X is a subset whose points are pairwise separated with distance at least $\epsilon $ , and such that every point $x\in X$ is at distance at most $\epsilon $ to a point of F.

Lemma 6.6. For any connected semisimple Lie subgroup $K < \mathrm {SO}(n-1)$ , let $F\subset K$ be a $\delta $ -net of K that generates a dense subgroup of K. There is a sufficiently small $\epsilon _0>0$ such that for any $\epsilon <\epsilon _0$ every $\epsilon $ -perturbation of F generates a subgroup of $\mathrm {SO}(n-1)$ whose closure contains a conjugate $h_\epsilon K h_\epsilon ^{-1}$ of K. Moreover, $h_\epsilon $ can be chosen so that $h_\epsilon \to e$ as $\epsilon \to 0$ .

Proof. First suppose there is no such $\epsilon _0$ . Then there is a sequence $\epsilon _n\to 0$ and an $\epsilon _n$ -perturbation $F_{\epsilon _n}\subset \mathrm {SO}(n-1)$ of F, with respect to the Hausdorff metric on closed subsets, such that the closure of the group generated by $F_{\epsilon _n}$ does not contain a conjugate of K. Let $H_n=\overline {\langle F_{\epsilon _n} \rangle }^0$ be the identity component of the closure of the group generated by $F_{\epsilon _n}$ . Consider the Levi decomposition of the connected Lie group $H_n$ , namely $H_n=H_n^{ss} H_n^{a}$ where $H_n^{ss}$ is maximal semisimple and $H_n^{a}$ is a closed connected solvable normal subgroup (see [Var84, Theorem 3.18.13] and [Gor94, Theorem 1.4.3]). Since $H_n^a$ is compact connected and solvable, it is an abelian torus. The automorphism group of this torus is discrete. Since $H_n$ is connected the image of the conjugation action $H_n\to \mathrm {Aut}(H_n^a)$ is trivial. Note also that $H^{ss}\cap H^a$ is semisimple and therefore trivial. This implies that $H_n$ is a group product of $H_n^{ss}$ and $H_n^a$ .

Since the topology induced by the Hausdorff metric on compact subsets of a compact space is compact, we may pass to a subsequence such that $H_n$ converges to $H=\lim _{n} H_n$ . Let $H^{ss}=\lim _{n} H^{ss}_n$ and $H^a=\lim _{n}H^a_n$ . The limit of abelian groups is abelian so $H^a$ is abelian. Hence $H=H^{ss} H^a$ since the limit of a product is a product. Since the generators converge, H therefore contains $\overline {\langle F \rangle }^0$ , but this is just K. The projection of K to $H^a$ is trivial since K is semisimple and $H^a$ is connected, and hence $H^{ss}$ contains K.

There are only finitely many conjugacy classes of connected semisimple subgroups of $\mathrm {SO}(n-1)$ . For each n the corresponding connected semisimple group $H_n^{ss}$ must belong to one of these. Since $H^{ss}$ contains K, the groups $H^{ss}_n$ must eventually belong to the same conjugacy class, also containing $H^{ss}$ . Therefore for all sufficiently large n, the group $H^{ss}_n$ must contain a conjugate of K. This contradiction establishes the existence of $\epsilon _0$ .

Moreover, we have shown that for every $\epsilon <\epsilon _0$ , if $F_\epsilon $ is an $\epsilon $ -perturbation of F generating a subgroup with closure $H_{\epsilon }<\mathrm {SO}(n-1)$ , then $H_{\epsilon }^{ss}={h_\epsilon } Q_\epsilon h_\epsilon ^{-1}$ for a compact semisimple group $Q_\epsilon $ containing K and belonging to one of finitely many conjugacy classes. In particular, $h_\epsilon K h_\epsilon ^{-1}< H_\epsilon ^{ss}$ . Moreover, $Q_\epsilon $ is close to $H^{ss}_\epsilon $ as $\epsilon \to 0$ , and so we may assume we have chosen $h_\epsilon $ so that $h_\epsilon \to e$ as $\epsilon \to 0$ .

Given two Riemannian metrics $g, g^{\prime } \in \operatorname {Sym}(T^{*}M \otimes T^{*}M)$ , there is a unique positive-definite vector bundle endomorphism A of $TM$ such that $g^{\prime }_x(v, w)=g_x(A_x v, w)$ at each point $x\in M$ . Let $q:TM\to TM$ be the bundle endomorphism $q:=A^{-\tfrac 12}$ . If f is a g-orthonormal frame then $q(f)$ is a $g^{\prime }$ -orthonormal frame. Thus q defines a smooth $\mathrm {SO}(n)$ -equivariant map of g-orthonormal frames to $g'$ -orthonormal frames. In other words, q induces a principal $\mathrm {SO}(n)$ -bundle isomorphism $q^F:(FM)_g\to (FM)_{g'}$ (see [BG92, Proposition 1]).

Now consider a $C^2$ perturbation $g_\epsilon $ through $C^\infty $ metrics of a negatively curved metric $g_0$ . As indicated above, there is a vector bundle isomorphism $q_\epsilon :TM\to TM$ which restricts to a fiber bundle isomorphism $q_\epsilon :(SM)_\epsilon \to (SM)_0$ between the unit tangent bundle $(SM)_\epsilon \to M$ for the metric $g_\epsilon $ and $(SM)_0\to M$ for the metric $g_0$ . Moreover, we have a principal $\mathrm {SO}(n)$ -bundle isomorphism $q_\epsilon ^F:(FM)_\epsilon \to (FM)_0$ between the orthonormal frame bundles $(FM)_\epsilon \to (SM)_\epsilon $ and $(FM)_0\to (SM)_0$ which fibers over $q_\epsilon $ . We may conjugate the geodesic flow $\phi ^\epsilon _t$ for $g_\epsilon $ by $q_\epsilon $ to obtain a flow $\widehat {\phi }^\epsilon _t:(SM)_0\to (SM)_0$ . Similarly, we may conjugate the frame flow $\Phi ^\epsilon _t$ for $g_\epsilon $ by $q_\epsilon ^F$ to obtain a flow $\widehat \Phi ^\epsilon _t:FM\to FM$ which extends $\widehat \phi ^\epsilon _t$ .

Using Remark 6.5, we may consider the transitivity group $H_v^\epsilon <\mathrm {Isom}(v^\perp )$ for the flow $\widehat {\Phi }^\epsilon _t$ . Also, since the map $q_\epsilon ^F$ is equivariant with respect to the right $\mathrm {SO}(n)$ action, the asymptotic holonomy group for $\Phi ^\epsilon _t$ is the same as $B^\infty _\epsilon $ up to conjugacy.

Lemma 6.7. Let $g_\epsilon $ be a $C^2$ perturbation through $C^\infty $ metrics of a negatively curved metric $g_0$ . Let $H_v^\epsilon $ be the transitivity group of a vector v for the metric $g_\epsilon $ , and suppose transitivity group $H_v^0$ for $g_0$ is connected and semisimple. Then for all sufficiently small $\epsilon $ , $H_v^\epsilon $ contains a conjugate of $H_v^0$ which limits to $H_v^0$ as $\epsilon \to 0$ .

Proof. In order to show there is a relation between $H_v^\epsilon $ for different $\epsilon $ , we use the description of the transitivity group. Namely, we may construct a net of elements in $H_v^\epsilon $ by following a sequence of paths along leaves of $W^u_{\widehat \Phi _t}$ and $W^s_{\widehat \Phi _t}$ starting from v which lie in a compact set. By the permanence theorem [HPS77, Theorem 6.8], the unstable and stable manifolds for $\widehat \Phi ^\epsilon _t$ , as a compact extension of $\widehat \phi ^\epsilon _t$ on $(SM)_0$ , vary continuously in the $C^1$ topology under $C^2$ perturbations. As these manifolds converge on compacta, using sufficiently long u–s paths, we obtain for any $\epsilon>0$ an $\epsilon $ -net in $H_v^\epsilon $ that is $\epsilon $ -close to an $\epsilon $ -net in $H_v^0$ .

By assumption, the transitivity group $H_v^0$ is connected and semisimple. Consequently, by Lemma 6.6, the transitivity group $H_v^\epsilon $ contains a conjugate of $H_v^0$ for sufficiently small perturbation parameter $\epsilon>0$ , and these subgroups converge to $H_v^0$ as $\epsilon \to 0$ .

The following lemma can be found in [Con08]; we provide a short proof for convenience. We say that a 2-frame $(u,w)$ has higher hyperbolic rank if the sectional curvatures satisfy $\kappa (\phi _t(u)\wedge \|_t w)\equiv -1$ for all $t\in \mathbb {R}$ where $\|_t$ is parallel translation along $\phi _t(u)$ .

Lemma 6.8. For a flow-invariant, full Liouville-measure subset of $v\in SM$ , $H_v$ sends higher hyperbolic rank 2-frames over v to higher hyperbolic rank 2-frames over v.

Proof. The set of 2-frames $(u,w)$ of higher hyperbolic rank remains invariant under parallel transport. Any 2-frame $(u',w')$ forward asymptotic to the parallel frame field of $(u,w)$ must therefore also have curvature limiting to $-1$ . If $(u',w')$ is recurrent then $(u',w')$ must have higher hyperbolic rank. If $(u'',w'')$ forward limiting to the parallel frame field of $(-u',w')$ is recurrent then it must similarly have higher hyperbolic rank. If the frame $(-u,\rho w)$ for $\rho \in H_u<\mathrm {Isom}(u^\perp )$ which forward limits to $(u'',w'')$ is recurrent then it has higher hyperbolic rank as well. Finally, the set of bi-recurrent vectors is a flow-invariant full Liouville measure set.

The following lemma is trivial, but we include it for completeness.

Lemma 6.9. If a sequence of subgroups $K_n<\mathrm {SO}(n)$ converges to $K < \mathrm {SO}(n)$ , and a sequence of vectors $v_n$ converges to a vector v, then the stabilizers in $K_n$ of vectors $v_n$ , $\operatorname {Stab}_{\mathrm {SO}(n)}(v_n)\cap K_n$ , converge to a subgroup of $\operatorname {Stab}_{\mathrm {SO}(n)}(v)\cap K$ .

Proof. If $k_n\in \operatorname {Stab}_{\mathrm {SO}(n)}(v_n)\cap K_n$ converges to $k\in K$ , then clearly $k\in \operatorname {Stab}_{\mathrm {SO}(n)}(v)$ .

Before proving Proposition 6.2, we describe the Brin–Pesin groups and their actions in the locally symmetric case.

Let $(M, g_0)$ be a closed complex, quaternionic or octonionic hyperbolic locally symmetric manifold. Hence $M=\Gamma \backslash \widetilde {M}$ for a cocompact lattice $\Gamma <G:=\operatorname {Isom}^0(\widetilde {M},g_0)$ , and let $K_x<G$ be the stabilizer of a point x and $L_v<K_x$ the stabilizer of a tangent vector $v\in T_xM$ . Observe that the left action of G on the full (oriented) orthonormal frame bundle decomposes $(F\widetilde {M})_{g_0}$ into G-orbits, which descend to closed subsets of $(FM)_{g_0}$ since G acts cocompactly on $(F\widetilde {M})_{g_0}$ . Since the frame flow corresponds to a right action by the one-parameter split Cartan subgroup, by Moore’s theorem [Zim84, Theorem 2.2.6] the quotients of the G orbits by $\Gamma $ coincide with the ergodic components of the frame flow on $(FM)_{g_0}$ for the lift of the Liouville measure. Since the intersection of the fiber over v with a G orbit coincides with the action of the stabilizer $L_v$ , which is faithful on frames, we have $H_v=L_v$ .

For the quaternionic hyperbolic space $\widetilde {M}=\mathbb {H}_{\mathbb {H}}^n$ we have $H_v=L_v\cong B^\infty =\mathrm {Sp}(n-1)\mathrm {Sp}(1)$ (here $\mathrm {Sp}(k):=\mathrm {Sp}(2k,{\mathbb {C}})\cap U(2k)$ ), and for the Cayley plane $\widetilde {M}=\mathbb {H}_{\mathbb {O}}^2$ we have $H_v=L_v\cong B^\infty =\mathrm {Spin}(8)\mathrm {Spin}(1)$ . Quaternionic symmetric space comes endowed with a quaternionic Kähler structure and the Cayley plane with an octonionic Kähler structure (see, for example, [Bes87]). These structures are parallel and thus preserved under the full holonomy group. Thus each higher hyperbolic rank 2-frame is fixed by $\mathrm {Sp}(n-1)$ and $\mathrm {Spin}(8)$ , respectively. However, the set of all such frames is acted on transitively by the $\mathrm {Sp}(1)$ and $\mathrm {Spin}(1)$ groups of dimension 3 and 7 respectively, since these groups act transitively on the perpendicular to v in the $\mathbb {H}$ -line (respectively, $\mathbb {O}$ -line) through v.

Proof of Proposition 6.2

By the hyperbolic rank condition, the statement for the complex hyperbolic case is trivial. We proceed by contradiction and suppose there is a sequence of metrics $g_{\epsilon _n}$ of higher hyperbolic rank with $\lVert g_{\epsilon _n}-g_0 \rVert _{C^2}<\epsilon _n$ where $\epsilon _n$ tends to $0$ , but whose hyperbolic ranks are all less than the hyperbolic rank of $g_0$ .

By Lemma 6.8, for almost every $v\in SM$ , $H_{\epsilon _n,v}$ preserves the hyperbolic rank 2-frames over v. Consider a sequence of generic 2-frames $(v_n,w_n)$ for $g_{\epsilon _n}$ that limit to a hyperbolic 2-frame $(v,w)$ for $g_0$ . By Lemma 6.7, if $\epsilon _n$ is sufficiently small, the asymptotic holonomy group $H_{\epsilon _n,v_n}$ for the perturbed metric $g_{\epsilon _n}$ contains a conjugate copy $L_{\epsilon _n}$ of $H_{v}$ , and these converge to $H_{v}$ as $\epsilon _n \to 0$ . Let $Q_{\epsilon _n}<L_{\epsilon _n}$ be the stabilizers in $L_{\epsilon _n}$ of $(v_n,w_n)$ . By Lemma 6.9, $Q_{\epsilon _n}$ converges to a subgroup of a conjugate of $\mathrm {Sp}(n-1)$ or $\mathrm {Spin}(8)$ , respectively. Hence if $\epsilon _n$ is sufficiently small and $(v_n,w_n)$ is chosen to be a hyperbolic 2-frame for $g_{\epsilon _n}$ then the $L_{\epsilon _n}$ -orbit of $(v_n,w_n)$ , consisting of hyperbolic 2-frames, has dimension at least as large as the dimension of the $H_{v}$ -orbit of $(v,w)$ . This contradicts the assumption that hyperbolic ranks of $g_{\epsilon _n}$ are less than that of $g_0$ .

Corollary 6.10. Let $(M, g_0)$ be a closed complex, quaternionic or octonionic hyperbolic locally symmetric manifold. Then there is an open $C^2$ neighborhood U of $g_0$ such that, for any $g\in U$ , if $(M,g)$ has higher hyperbolic rank and $\kappa _g\ge -1$ then the hyperbolic rank of every tangent vector to $(M,g)$ is exactly $1$ , $3$ or $7$ , respectively.

Proof. If v is a unit tangent vector of $(M,g_0)$ then $v^{\perp _{g_0}}=\widehat {\mathcal {E}}_{g_0}(v)\oplus \widehat {\mathcal {E}}_{g_0}(v)^{\perp _{g_0}}$ , where the sectional curvature $\kappa (v\wedge w)=-1$ for every $w\in \widehat {\mathcal {E}}_{g_0}(v)$ and $\kappa (v\wedge u)=-\tfrac 14$ for every $u\in \widehat {\mathcal {E}}_{g_0}(v)^{\perp _{g_0}}$ .

Similarly, if v is a unit tangent vector of $(M,g)$ , we also have $v^{\perp _{g}}=\widehat {\mathcal {E}}_{g}(v)\oplus \widehat {\mathcal {E}}_{g}(v)^{\perp _{g}}$ , where the sectional curvature $\kappa (v\wedge w)=-1$ for every $w\in \widehat {\mathcal {E}}_{g}(v)$ . Since g is sufficiently $C^2$ -close to $g_0$ , the continuity of curvatures in the $C^2$ topology of Riemannian metrics implies that $\dim (\widehat {\mathcal {E}}_{g}(v))\le \dim (\widehat {\mathcal {E}}_{g_0}(v))$ for every v. In particular, the hyperbolic rank of every tangent vector of $(M,g)$ is at most the hyperbolic rank of $(M,g_0)$ .

On the other hand, by Proposition 6.2, the hyperbolic rank of $(M,g)$ is not smaller than the hyperbolic rank of $(M,g_0)$ . By item (1) of Lemma 6.1, the hyperbolic rank of every tangent vector of $(M,g)$ is exactly the hyperbolic rank of $(M,g_0)$ .

6.2 Quaternionic and octonionic hyperbolic local rigidity

We are now ready to prove the main result about higher hyperbolic rank stated in the introduction.

Theorem 1.14. Let $(M, g_0)$ be a closed quaternion or octonionic hyperbolic locally symmetric manifold. Then there is an open $C^3$ neighborhood U of $g_0$ such that, for any $g\in U$ , if $(M,g)$ has higher hyperbolic rank and $\kappa _g\ge -1$ then $(M,g)$ is locally symmetric and isometric to $(M,g_0)$ .

Proof of Theorem 1.14

By Corollary 6.10, we may choose U such that the hyperbolic rank of every tangent vector of $(M,g)$ is the same as the hyperbolic rank of $(M,g_0)$ .

Next, let $J_{\xi }(t)$ denote the unstable Jacobi field along $\phi _t(v)$ with initial vector $\xi \in \widehat {\mathcal {E}}(v)$ . By Lemma 6.1, we can make the following observations about Oseledets spaces:

$$ \begin{align*} E_1^u(v)=\{ (\xi, J_{\xi}'(0)): \xi\in \widehat{\mathcal{E}}(v) \} \quad\text{and}\quad E_{\mathrm{slow}}^u(v)=\{ (\xi, J_{\xi}'(0)): \xi\in \widehat{\mathcal{E}}(v)^\perp \}. \end{align*} $$

By Corollary 6.10, $\dim E_1^u$ is constant, and by Lemma 6.1, $E_1^u$ is a globally $C^\infty $ distribution on $SM$ . In particular, $E_1^u$ is uniformly $C^\infty $ along $W^u$ for each such metric g. Moreover, as the curvature operator, and hence Jacobi fields, vary $C^1$ in the $C^3$ topology on smooth metrics, $E_1^u$ and its first derivatives along $W^u$ vary continuously in the metric parameter with respect to the $C^3$ topology on smooth metrics. By [CNS20, Lemma 4.3], $E_1^u$ and $E_{\mathrm {slow}}^u$ are perpendicular with respect to the Sasaki metric for g. Hence the dominated splitting $E^u=E_1^u\oplus E_{\mathrm {slow}}^u$ is uniformly $C^\infty $ along unstable leaves. By the permanence theorem [HPS77, Theorem 6.8], $E_{\mathrm {slow}}^u$ has $C^1$ dependency on g as g varies in the $C^3$ topology. Therefore by Theorem 1.9, M is symmetric for sufficiently $C^3$ nearby smooth metrics.

Remark 6.11. One can show that $E^u_1$ varies continuously in the $C^1$ topology as the metric g varies in the $C^2$ topology. However, since $E^u$ may not vary continuously in the $C^1$ topology as the metric g varies in the $C^2$ topology, we are not able to conclude that $E^u_{\mathrm {slow}}$ varies $C^1$ in this topology. The latter is necessary to obtain the isomorphism of tangent cones. Hence we require the metrics to be $C^3$ -close.

We next observe that the higher hyperbolic rank condition in the last theorem may be replaced by a condition on Lyapunov exponents.

Corollary 6.12. In Theorem 1.14, the conclusion still holds if the higher hyperbolic rank assumption is replaced with the assumption that some ergodic measure of full support has a Lyapunov exponent equal to 1.

This follows from the following lemma, whose proof is essentially that of [CNS20, Corollary 1.4]. (See also [Con03, Theorem 1.3] for the case $\kappa \leq -1$ .)

Lemma 6.13. Let M be a closed Riemannian manifold with $\kappa \geq -1$ . If, with respect to the geodesic flow on $SM$ , some ergodic measure of full support has a Lyapunov exponent equal to $1$ , then M has higher hyperbolic rank.

6.3 Measure of maximal entropy for perturbations

For the case of complex hyperbolic metrics, we are not able to obtain full local rigidity. Indeed, our methods require that the tangent cones of unstable leaves are asymmetric, which the Heisenberg group is not. However, we still obtain equality of the Liouville measure and the Bowen–Margulis measure (the unique measure of maximal entropy) for a higher-rank perturbation. The precise result of this subsection is Theorem 1.15 from the introduction.

Theorem 1.15. Let $(M,g_0)$ be a closed complex hyperbolic manifold. There is an open neighborhood U of $g_0$ in the $C^3$ topology among $C^\infty $ metrics such that if $g\in U$ and $(M,g)$ has higher hyperbolic rank and sectional curvature $\kappa \ge -1$ then the Liouville measure on $SM$ coincides with the (unique) measure of maximal entropy for the geodesic flow of g on $SM$ .

Proof. We first show that the tangent cone $TC_vW^u(v)$ exists for every $v\in SM$ .

By Corollary 6.10, $E_1^u\subset E^u$ is one-dimensional everywhere. By (4) of Lemma 6.1, $E_1^u$ forms a smooth distribution along each leaf of the foliation $W^u$ . By (5) of Lemma 6.1, the lift $E_{\mathrm {slow}}^u=E_{<1}^u$ of $\widehat {\mathcal {E}}^\perp $ to $E^u$ is the direct sum of unstable Lyapunov spaces of exponents different from one. Therefore, $E_{\mathrm {slow}}^u$ is uniformly $C^\infty $ along $W^u$ since it is orthogonal to $E^u_1$ in $E^u$ with respect to the Sasaki metric.

By the same argument used in the proof of Theorem 1.14, $E_{\mathrm {slow}}^u$ has $C^1$ dependency on g as g varies in the $C^3$ topology. In particular, $E^u_{\mathrm {slow}}$ is generic everywhere on $SM$ . Also, by Theorem 3.2, the tangent cone $TC_vW^u _g$ exists everywhere.

By Corollary 6.10, the horizontal distribution of $TC_vW^u_g(v)$ has codimension one. Hence $TC_vW^u_g(v)$ is a 2-step Carnot nilpotent Lie group with one-dimensional center for every $v\in SM$ . By Lemma 5.5, $TC_vW^u_g(v)$ depends continuously on the metric g in the $C^3$ topology. Recall that the Lie bracket induces a bilinear form $[\cdot ,\cdot ]:\mathfrak {n}_v^0\times \mathfrak {n}_v^0\to \mathfrak {n}_v^1$ between the first and second levels $\mathfrak {n}_v^0$ and $\mathfrak {n}_v^1$ of the Lie algebra $\mathfrak {n}_v$ of $TC_vW^u_g(v)$ . As it is non-degenerate for $g_0$ , it will also be non-degenerate for g sufficiently close to $g_0$ . Consequently, as the center is one-dimensional and there is only one, up to isomorphism, non-degenerate skew symmetric bilinear form on $\mathfrak {n}_v^0$ for every $v\in SM$ , $TC_vW^u(v)$ is isomorphic as a graded nilpotent group to the Heisenberg group $H^{2k-1}$ where ${\dim M=2k}$ .

By Theorem 1.3, if $v\in SM$ is a periodic vector then the sum of the positive Lyapunov exponents (with multiplicity) at v is k. By [Kal11, Theorem 1.4], it follows that with respect to any ergodic invariant probability measure, the entropy of geodesic flow is bounded above by k. On the other hand, by Pesin’s entropy formula, the entropy of the geodesic flow with respect to Liouville measure (rescaled to have total volume one) is k. Therefore the (rescaled) Liouville measure is the measure of maximal entropy.

Remark 6.14. Local rigidity for higher hyperbolic rank metrics near the complex hyperbolic one would follow from Katok’s well-known entropy conjecture stating that the Liouville measure has maximal entropy precisely for locally symmetric manifolds among closed negatively curved manifolds. Alternately, and possibly more simply, it would follow from strict convexity of the difference of topological and measure-theoretic entropy near the complex hyperbolic metric. For real hyperbolic metrics, this was proven by Flaminio [Fla95, Theorem A].

Remark 6.15. As in Corollary 6.12, we may replace the condition on higher hyperbolic rank in Theorem 1.15 with the condition that there is a Lyapunov exponent equal to 1 for some invariant ergodic measure of full support.