The n-Category Café

The parameter $q$ determines how much $D_q^Z(p)$ counts the very “rare” points (those for which $p(x)$ is very small). An interesting question from an ecological point of view is, given $X$ and $Z$, which probability distribution $p$ maximizes the diversity $D_q^Z(p)$? It turns out that the answer is independent of $q$. Moreover, if $X$ is a metric space and $Z(x,y) = e^{-d(x,y)}$, this maximum diversity $$D(X) := \max_p D_q^Z(p)$$ is an isometric invariant closely related to the magnitude of $X$. It also extends in a natural way to compact metric spaces.

Independently, David Bryant and Paul Tupper defined a diversity on a set $X$ to be a $[0,\infty)$-valued function $\delta$ on the finite subsets of $X$ which satisfies:

• $\delta(A) = 0$ if $A$ has at most one element, and

• $\delta(A\cup B) \le \delta(A \cup C) + \delta(C \cup B)$ whenever $C \neq \emptyset$.

I will refer to a diversity in this sense as a BT diversity. If $\delta$ were defined only on sets with at most two elements, this would amount to the definition of a metric. In fact, if $d$ is a metric on $X$, then $$\delta(A) = diam (A) := \max_{a,b \in A} d(a,b)$$ defines a BT diversity on $X$, so BT diversities are actually a generalization of metrics.

Here as well, the motivation for the name “diversity” comes from an example in theoretical ecology: suppose $X$ is a set of species in a phylogenetic tree $T$. Define $\delta(A)$ to be the length of the smallest subtree of $T$ containing $A$. Then $\delta$ is a BT diversity, known in the literature as phylogenetic diversity. However, just as with the maximum diversity discussed above, most of the subsequent work on BT diversities has focused on geometric examples.

So we now have two seemingly quite different geometric notions, introduced about the same time, going by strikingly similar names for conceptually similar reasons. One can’t help wondering, do they have something to do with each other? In particular, could maximum (LC) diversity be an example of a BT diversity?

In a new paper with Gautam Ashwarya, Dongbin Li, and Mokshay Madiman, we show that, after a minor tweak, maximum diversity does give rise to a BT diversity. The minor tweak is necessary to handle the first condition in the definition of BT diversity: if $X$ is a metric space and $x \in X$, it’s easy to check that $D(\{x\}) = 1$, whereas a BT diversity must satisfy $\delta(\{x\}) = 0$. This can be dealt with in the simplest imaginable way:

Theorem 1 Let $X$ be a metric space. For each nonempty finite $A \subseteq X$ set $\delta(A) = D(A) - 1$, and define also $\delta(\emptyset) = 0$. Then $\delta$ is a BT diversity on $X$.

(In the paper itself, we adopt the term complexity when referring to the quantities $\log D_q^Z(p)$ and $\log D(X)$, and state most of the results in terms of complexity instead of maximum diversity; we further deduce from Theorem 1 that the complexity $log D(X)$ is also a BT diversity. This terminology is used partly to cut down on the potential confusion created by using “diversity” in multiple ways. It also alludes to the relationship between $\log D_q^Z(p)$ and Rényi entropy, which is widely understood as a measure of “complexity”. Further connections between LC complexity and Rényi entropy are the subject of forthcoming work that I hope to be able to tell you more about soon! But for the remainder of this blog post I’ll stick to the maximum diversity formulation.)

Interestingly, maximum diversity has some properties that are quite nice and natural, but turn out to make it intriguingly different from the heretofore most thoroughly studied BT diversities. For example, $D = 1 + \delta$ has the following subadditivity property, which is not shared by the functional $1 + diam$:

Theorem 2 Let $X$ be a metric space, and let $A_1, \ldots, A_n \subseteq X$ be compact subsets. Then $$D\left(\bigcup_{i=1}^n A_i \right) \le \sum_{i=1}^n D(A_i).$$

Maximum diversity actually satisfies a much stronger property called fractional subadditivity, which arises naturally in inequalities for entropy. Another special case of fractional subadditivity is the following.

Theorem 3 Let $X = \{x_1, \ldots, x_n\}$ be a finite metric space. Then $$\frac{D(X)}{n} \le \frac{1}{n} \sum_{i=1}^n \frac{D(X \setminus \{x_i\})}{n-1}.$$

Theorem 3 can be interpreted as saying that the “complexity per element” of $X$ is at most the average complexity per element of a randomly chosen subset of cardinality $n-1$. This captures the natural intuition that as the size of a metric space increases, its complexity per element decreases.

In the setting of $\mathbb{R}^n$, many examples of BT diversities are homogeneous, in the sense that $\delta(\lambda A) = \lambda \delta(A)$ for all $\lambda \ge 0$ and nonempty finite $A \subseteq \mathbb{R}^n$, and either sublinear, meaning homogeneous and also satisfying $$\delta(A + B) \le \delta(A) + \delta(B),$$ or else linear, where we have equality in the condition above. For example, the diameter is a sublinear diversity. (Diversities with these properties are the focus of a recent work by Bryant and Tupper.)

By contrast, maximum diversity has no simple homogeneity property; in fact its complex behavior with respect to scaling is part of what gives it such rich geometric interest. And at least in one dimension, the diversity $\delta = \log D$ satisfies the following superlinearity properties.

Theorem 4 Let $\delta$ be the diversity $\delta = \log D$ defined on compact subsets of $\mathbb{R}$. Then $$\delta(A + B) \ge \delta(A) + \delta(B)$$ and $$\delta(\lambda A + (1-\lambda)B) \ge \lambda \delta(A) + (1-\lambda) \delta(B)$$ for every $0 \le \lambda \le 1$ and nonempty compact $A,B \subseteq \mathbb{R}$.

The first inequality in Theorem 4 can be regarded as a generalization of the Cauchy–Davenport inequality in $\mathbb{R}$, and the second as a version of the Brunn–Minkowski inequality in $\mathbb{R}$. (In fact, since Lebesgue measure can be recovered from maximum diversity, it implies the Brunn–Minkowski inequality in $\mathbb{R}$.) It is an open question, for which we know some partial results, whether Theorem 4 can be extended to higher dimensions.

In conclusion, our results make (at least) the following points:

• The seemingly independent mathematical notions of diversity introduced by Leinster and Cobbold on the one hand, and Bryant and Tupper on the other hand, are actually closely connected.

• Maximum diversity, in the sense of LC diversities, leads to a geometrically interesting example of a BT diversity whose behavior is quite different from many of the previously studied examples of BT diversities.

• Maximum diversity, at least in certain contexts, satisfies a number of inequalities which extend important classical inequalities, and it would be especially interesting to push this line of inquiry further.

Please read the paper itself for more detail and other remarks (it’s short!).