Planet Musings

Quanta magazine has just published a feature on Tai-Danae Bradley and her work, entitled

Where Does Meaning Live in a Sentence? Math Might Tell Us.

The mathematician Tai-Danae Bradley is using category theory to try to understand both human and AI-generated language.

It’s a nicely set up Q&A, with questions like “What’s something category theory lets you see that you can’t otherwise?” and “How do you use category theory to understand language?”

Particularly interesting for me is the part towards the end where Bradley describes her work with Juan Pablo Vigneaux on magnitude of enriched categories of texts.

Today I learned about ‘rabbit starvation’ and how Neanderthals avoided it.

When you’re a hunter-gatherer and it’s winter, you may try to survive by eating only meat—like rabbits, but also deer and other game. But this gives you too much protein and not enough carbohydrates and fat: most of this meat is very lean. If you eat enough lean meat to get all the calories you need, you can die from an overdose of protein! It’s called ‘protein toxicity’.

Hunter-gatherers in this situation sometimes throw away the ‘steaks’ and ‘roasts’—the thighs and shoulders of the animals they kill—or feed them to their dogs. They need fat to survive! So they focus on eating the fatty parts, including bone marrow.

(Now I better understand how we domesticated wolves. In the winter we didn’t just give them scraps. We gave them steaks!)

In some cultures, while the men are out hunting, the women spend time making bone grease. This takes a lot of work. They take bones and break them into small pieces with a stone hammer. They boil them for several hours. The fat floats to the top. Then they let the water cool and skim off the fat.

There’s been evidence for people doing this as far back as 28,000 BC. But now some scientists have found a Neanderthal ‘bone grease factory’ that’s 125,000 years old!

This was during the last interglacial, the Eemian, in Germany. In a site near a lake, called Neumark-Nord, Neanderthals killed a lot of bison, horses and deer and crushed their bones, leaving behind tens of thousands of small bone fragments.

• Lutz Kindler, Sabine Gaudzinski-Windheuser, Fulco Scherjon, Alejandro Garcia-Moreno, Geoff M. Smith, Eduard Pop, John D. Speth, and Wil Roebroeks, Large-scale processing of within-bone nutrients by Neanderthals, 125,000 years ago, Science Advances 11 27 (2025), eadv1257.

There is an informal meeting at MPIA every Wednesday regarding binary stars, with a bit of a focus on massive binaries. Today there was a very nice presentation by Jakob Stegmann (MPA) about some anomalies among the (only six) black-hole–neutron-star binaries discovered by NSF LIGO. He showed the example of GW 200105, which shows a large eccentricity (0.15-ish). This eccentricity is very hard to explain, given how the inspirals evolve as they radiate. But the analysis of the eccentricity (from perhaps this paper) is Bayesian, so it isn't clear how much the eccentricity result is forced by the data and how much is forced by the prior over nuisance parameters. That's one of the main points of my forthcoming paper on measurement. I think maybe I should just re-analyze this one with a profile likelihood. I hope the data and code are public!

guest post by Bruce Bartlett

Stellenbosch University is hiring!

The Mathematics Division at Stellenbosch University in South Africa is looking to hire a new permanent appointment at Lecturer / Senior Lecturer level (other levels may be considered too under the appropriate circumstances).

Preference will be given to candidates working in number theory or a related area, but those working in other areas of mathematics will definitely also be considered.

The closing date for applications is 30 April 2025. For more details, kindly see the official advertisement.

Consider a wonderful career in the winelands area of South Africa!

How do you count rooted planar $n$-ary trees with some number of leaves? For $n = 2$ this puzzle leads to the Catalan numbers. These are so fascinating that the combinatorist Richard Stanley wrote a whole book about them. But what about $n \gt 2$?

I’ll sketch one way to solve this puzzle using generating functions. This will give me an excuse to talk a bit about something called ‘Lagrange inversion’.

By the way, a mentally ill person may post lots of strange comments here under various names — perhaps even under my name. Please don’t engage with him, or even discuss this issue here, since it will only worsen his illness. Just ignore him and talk about the math.

You can define a rooted planar binary tree recursively by this equation:

$$B \cong x + B^2$$

This means “to make a set into the leaves of a binary tree, it should either be the one-element set or the leaves of a pair of binary trees”. You can then solve this using the quadratic formula and get

$$B \cong \frac{1 - \sqrt{1 - 4x}}{2} = x + x^2 + 2 x^3 + 5 x^4 + 14 x^5 + \cdots$$

where the numbers here are the Catalan numbers. For example, there are 5 binary trees with 4 vertices. For more details and more rigor go here.

We can define rooted planar $n$-ary trees by a similar equation:

$$T \cong x + T^n$$

Bu how do we solve this when $n \gt 2$? Even for $n = 3$ or $n = 4$, where we could use the cubic or quartic equation, we don’t really want to.

The trick is to use Lagrange inversion. Given a formal power series

$$\displaystyle{ f(z) = \sum_{k=1}^\infty f_k \frac{z^k}{k!} }$$

normalized with $f_1 = 1$ to make things simple, Lagrange inversion tells us a formal power series $g$ that’s the inverse of $f$ with respect to composition:

$$g(f(z)) = z, \qquad f(g(z)) = z$$

Here’s how it works. Let’s write

$$\displaystyle{ g(z) = \sum_{j=1}^\infty g_j \frac{z^j}{j!} }$$

Then $g_1 = 1$, and the Lagrange inversion theorem says that for $j \ge 2$ we have

$$\displaystyle{ g_j = \sum_{k=1}^{j-1} (-1)^k \; j^{\overline{k}} \; B_{j-1,k}(\hat{f}_1,\hat{f}_2,\ldots,\hat{f}_{j-k}) }$$

where

$$\displaystyle{ \hat{f}_k = \frac{f_{k+1}}{(k+1)} }$$

and the rising powers $j^{\overline{k}}$ are defined by

$$\displaystyle{ j^{\overline{k}} = j(j+1)\cdots (j+k-1) }$$

But the main ingredient in this formula is the Bell polynomials $B_{j,k}$. These are named after Eric Temple Bell, author of a science fiction series about a planet where mathematics is done only by men.

It seems easiest to explain the Bell polynomials with an example. $B_{j,k}$ is a polynomial in $j-1$ variables. It keeps track of all the ways you can partition an $j$-element set into $k$ disjoint nonempty subsets, called blocks. For example:

$$B_{6,2}(x_1,x_2,x_3,x_4,x_5)=$$ $$6x_5x_1+15x_4x_2+10x_3^2$$

This says that if we partition a 6-element set into 2 blocks there are:

• 6 ways to partition it into a block of size 5 and a block of size 1
• 15 ways to partition it into a block of size 4 and a block of size 2
• 10 ways to partition it into two blocks of size 3.

and those are all the ways!

So, if you know all the Bell polynomials, you know how many ways there are to partition any finite set into some chosen number of blocks of some chosen sizes.

The formula for Lagrange inversion is intimidating yet intriguing, and that’s what I really want to understand. But for now let’s just apply it to count $n$-ary trees. We’re trying to solve

$$T = x + T^n$$

or in other words

$$x = T^n - T$$

for $x$. So we make up a function

$$f(z) = z^n - z$$

and we seek the inverse power series $g$: this will give us $T$ as a power series in $x$.

The first coefficient of $f$ is $-1$, not $1$, so we’ll need to tweak the formula for Lagrange inversion. Luckily this will just get rid of the sign $(-1)^k$ that appeared in that formula.

Actually I’ll skip the detailed calculation, which is much less fun to read than to do yourself. The main point is that Lagrange inversion does the job. I’ll just give you the answer:

$$\displaystyle{ g(z) = \sum_{k=0}^\infty \binom{n k}{k} \frac{z^{(n-1)k+1} }{(n-1)k+1} }$$

So, the number of rooted planar $n$-ary trees with $(n-1)k + 1$ leaves should be

$$\binom{n k}{k} \frac{1}{(n-1)k+1}$$

As a sanity check, note that an $n$-ary tree always has $(n-1)k + 1$ leaves for some natural number $k \ge 0$, because it can either have $1$ leaf (the root), or we can stick a sprout with $n$ leaves on an existing leaf, thus adding $n-1$ new leaves.

Also note that when $n = 2$ we get our friends the Catalan numbers!

So this is pretty cool, and it raises tons of interesting questions, mostly about the deep inner meaning of Lagrange inversion. Richard Stanley wrote about this in Section 5.4 of the second volume of Enumerative Combinatorics, André Joyal wrote about it in Theorem 2 of Une théorie combinatoire des séries formelles (with a partially finished English translation here), and Flajolet and Sedgewick wrote about it in Appendix A.6 of Analytic Combinatorics. So there’s a lot of material to read! But I find all these discussions puzzling, so I’m trying to dig deeper and find an explanation that’s easier to grasp. Luckily Todd Trimble knows a lot about this subject, and it’s very beautiful! So stay tuned.

Despite the “2” in the title, you can follow this post without having read part 1. The whole point is to sneak up on the metricky, analysisy stuff about potential functions from a categorical angle, by considering constructions that are categorically reasonable in their own right. Let’s go!

Let $V$ be a monoidal category, which I’ll assume is symmetric, closed, complete or cocomplete as necessary. Take a pair of $V$-categories $A$ and $B$, and a $V$-functor

$$F: A \to B.$$

Then we get a new functor

$$N_F: B \to [A^{op}, V]$$

defined by

$$(N_F(b))(a) = B(F a, b).$$

This is sometimes called the nerve functor induced by $F$, because of the first example in the following list.

Examples   In the first few examples, I’ll take the enriching category $V$ to be $Set$.

• Let $F$ be the inclusion $\Delta \hookrightarrow Cat$, which takes a finite nonempty totally ordered set $[n] = \{0, \ldots, n\}$ and realizes it as a category in the usual way. Then $N_F: Cat \to [\Delta^{op}, Set]$ sends a small category $D$ to its nerve, which is the simplicial set whose $n$-simplices are paths in $D$ of length $n$.

• Let $F$ be the functor $\Delta \to Top$ that sends $[n]$ to the topological $n$-simplex $\Delta^n$. Then $N_F: Top \to [\Delta^{op}, Set]$ sends a topological space $X$ to its “singular simplicial set”, whose $n$-simplices are the continuous maps $\Delta^n \to X$.

• Let $F: A \hookrightarrow B$ be an inclusion of partially ordered sets, viewed as categories in the usual way. Then $N_F : B \to [A^{op}, Set]$ is given by

  $$(N_F(b))(a) = \begin{cases} 1 &\text{if}   b \leq a \\ 0 &\text{otherwise}. \end{cases}$$

  For example, let $B$ be the set $\mathbb{Z}$ of integers, ordered by divisibility. Let $A \subseteq \mathbb{Z}$ be the set of (positive) primes. Then $N_F$ is the functor $\mathbb{Z} \to Set^{\{primes\}}$ that sends $n \in \mathbb{Z}$ to

  $$([p | n])_{\text{primes}   p}$$

  where $[p|n]$ is Iverson notation: it’s $1$ if $p$ divides $n$ and $0$ otherwise.

• If $F$ has a right adjoint $G$, then $N_F(b) = A(-, G b): A^{op} \to V$. In particular, $N_F(b)$ is representable for each $b \in B$.

• Take the inclusion $Field \hookrightarrow Ring$, where $Ring$ is the category of commutative rings. Now take opposites throughout to get $F: Field^{op} \hookrightarrow Ring^{op}$. The resulting nerve functor

  $$N_F: Ring^{op} \to [Field, Set]$$

  sends a ring $R$ to $Ring(R, -)|_{Field}$ (by which I mean the restriction of $Ring(R, -): Ring \to Set$ to the category of fields). More explicitly,

  $$N_F(R) = \sum_{p \in Spec(R)} Field(Frac(R/p), -).$$

  Here $Spec(R)$ is the set of prime ideals $p$ of $R$, and $Frac(R/p)$ is the field of fractions of the integral domain $R/p$. In particular, $N_F(b)$ is a coproduct of representables for each $b \in B$. (In the terminology of Diers, one therefore says that $Field \hookrightarrow Ring$ is a “right multi-adjoint”. A closely related statement is that it’s a parametric right adjoint. But that’s all just jargon that won’t matter for this post.)

Now let’s consider $V = (\mathbb{R}^+, \geq)$ with its additive monoidal structure, so that $V$-categories are generalized metric spaces and $V$-functors are the functions $f$ between metric spaces that are variously called short, distance-decreasing, contractive, or 1-Lipschitz: $d(f(a), f(a')) \leq d(a, a')$ for all $a, a'$. I’ll just call them “maps” of metric spaces.

• Take a map $f: A \to B$ of metric spaces. The induced map $N_f: B \to [A^{op}, \mathbb{R}^+]$ is given by

  $$(N_f(b))(a) = d(f(a), b).$$

  Unlike in the first post, I’m not going to assume here that our metric spaces are symmetric. so the “op” on the $A$ matters, and $d(a, f(b)) \neq d(f(b), a)$ in general.

  We’ll be particularly interested in the case where the $V$-functor $F$ is full and faithful. Most of the examples above have this property. In the metric case, $V = \mathbb{R}^+$, being full and faithful means being distance-preserving, or in other words, an inclusion of a metric subspace. In that case, it’s normal to drop the $f$. So we’d usually then write

  $$(N_f(b))(a) = d(a, b).$$

Next I’ll define the potential function of a $V$-functor. For this we need some of the apparatus of magnitude. Roughly speaking, this means we have a notion of the “size” of each object of our base monoidal category $V$.

More exactly, we take a field $k$ and a function $|\cdot|: ob(V) \to k$, to be thought of as assigning to each object $S$ of $V$ its size ${|S|} \in k$. And to make everything work, we assume our size function $|\cdot|$ has reasonable properties:

$$S \cong T \implies |S| = |T|, \quad |S \otimes T| = |S| \cdot |T|, \quad |I| = 1,$$

where $\otimes$ is the monoidal product on $V$ and $I$ is its unit.

The basic definitions are these. A weighting on a finite $V$-category $A$ is a function $w_A : ob(A) \to k$ such that

$$\sum_{b \in A} |A(a, b)| w_A(b) = 1$$

for all $a \in A$, and a coweighting $w^A$ on $A$ is a weighting on $A^{op}$. Although it’s not true that every finite $V$-category has a unique weighting, it’s a harmless enough assumption that I’ll make it here. The magnitude of a finite $V$-category $A$ is

$$|A| = \sum_{a \in A} w_A(a) = \sum_{a \in A} w^A(a) \in k.$$

It’s a tiny lemma that the total weight $\sum_{a \in A} w_A(a)$ is equal to the total coweight $\sum_{a \in A} w^A(a)$, so you can equivalently define the magnitude to be one or the other.

There’s a whole lot to say about why this is a worthwhile definition, and I’ve said it in detail many times here before. But here I’ll just say two short things:

• Taking $V = FinSet$, $k = \mathbb{Q}$, and $|\cdot|$ to be cardinality, the magnitude of a finite category is equal to the Euler characteristic of its classifying space, under suitable hypotheses guaranteeing that the latter is defined.

• Take $V = \mathbb{R}^+$, $k = \mathbb{R}$, and $|\cdot|$ to be $s \mapsto e^{-s}$ (because that’s just about the only function converting the tensor product of $V$ into addition, as our size functions are contractually obliged to do). Then we get a notion of the magnitude of a metric space, a real-valued invariant that turns out to be extremely interesting.

That’s the magnitude of enriched categories. But we can also talk about the magnitude of enriched presheaves. Take a finite $V$-category $A$ and a $V$-presheaf $X: A^{op} \to V$. Its magnitude is

$$|X| = \sum_{a \in A} w^A(a) |X(a)| \in k.$$

When I wrote about the magnitude of functors before, I concentrated on covariant functors $A \to V$, which meant that the weights in the weighted sum that defines $|X|$ were, well, the weights $w_A(a)$. But since we’ve now changed $A$ to $A^{op}$, the weights have become coweights.

Let me briefly recall why the magnitude of presheaves is interesting, at least in the case $V = FinSet$:

• If our presheaf $X$ is a coproduct of representables then $|X|$ is $|colim(X)|$, the cardinality of the colimit of $X$.

• The magnitude of presheaves generalizes the magnitude of categories. The magnitude of a category $A$ is equal to the magnitude of the presheaf $A^{op} \to Set$ with constant value $1$.

• In the other direction, the magnitude of categories generalizes the magnitude of presheaves: $|X| = |\mathbb{E}(X)|$ for all presheaves $X$. Here $\mathbb{E}(X)$ means the category of elements of $X$, also called the (domain of the) Grothendieck construction.

  It’s enlightening to think of this result as follows. If we extend the codomain of $X$ from $Set$ to $Cat$ then $\mathbb{E}(X)$ is the colax colimit of $X: A^{op} \to Cat$. So $|X| = |colaxcolim(X)|$. You can compare this to the more limited result that $|X| = |colim(X)|$, which only holds when $X$ has the special property that it’s a coproduct of representables.

  There’s also another closely related result, due to Ponto and Shulman: under mild further hypotheses, $|X| = \chi(hocolim(X))$. Mike and I had a little discussion about the relationship here.

  Now here’s something funny. Which do you think is a more refined invariant of a presheaf $X: A^{op} \to FinSet$: the magnitude $|X|$, which is equal to $|colaxcolim(X)|$ and $\chi(hocolim(X))$, or the cardinality $|colim(X)|$ of the strict colimit?

  From one perspective, it’s the magnitude. After all, we usually think of lax or pseudo 2-dimensional things as more subtle and revealing than strict 1-dimensional things. But from another, it’s the cardinality of the strict colimit. For instance, if $A$ is the one-object category corresponding to a group $G$ then $X$ is a right $G$-set, the cardinality of the colimit is the number of orbits (an interesting quantity), but the magnitude is always just the cardinality of the set $X$ divided by the order of $G$ (relatively boring and crude).

  In this post I’ll keep comparing these two quantities. Whichever seems “more refined”, the comparison is interesting in itself.

(I stuck to the case $V = FinSet$ here, because for an arbitrary $V$, there isn’t a notion of “colimit” as such: we’d need to think about weighted colimits. It’s also harder to say what the category of elements or homotopy colimit of an enriched presheaf should be.)

Now here’s something new. The potential function of a $V$-functor $F: A \to B$ is the function

$$h_F: \ob(B) \to k$$

defined by

$$h_F(b) = |N_F(b)|$$

for all $b \in B$. Expanding out the definitions, this means that

$$h_F(b) = |A(F-, b)| = \sum_{a \in A} w^A(a) |B(F a, b)|,$$

where $w^A$ is the notation I introduced for the coweighting on $A$. If $F$ is full and faithful then $h_F \equiv 1$ on the set of objects in the image of $F$.

Examples   The definition I just gave implicitly assumes that $\ob(A)$ is finite. But I’ll relax that assumption in some of these examples, at the cost of some things getting less than rigorous.

Again, I’ll begin with some examples where $V = FinSet$.

• What’s the potential function of the inclusion $\Delta \hookrightarrow Cat$? To make sense of this, we need a coweighting on $\Delta$, and I’m pretty sure there isn’t one. So let’s abandon $\Delta$ and instead use its subcategory $\Delta_{inj}$, consisting of all the finite nonempty totally ordered sets $[n] = \{0, \ldots, n\}$ but only the injective order-preserving maps between them. This amounts to considering only face maps, not degeneracies.

  The coweighting on $\Delta_{inj}$ is $[n] \mapsto (-1)^n$. So, the potential function $h$ of $\Delta_{inj} \hookrightarrow Cat$ is given, on a category $D$, by

  $$h(D) = \sum_{n \geq 0} (-1)^n |Cat([n], D)|.$$

  Here $Cat([n], D)$ is the set of paths of length $n$ in $D$, which we often write as $D_n$. So

  $$h(D) = \sum_{n \geq 0} (-1)^n |D_n|.$$

  Even if the category $D$ is finite, this is a divergent series. But morally, $h(D)$ is the alternating sum of the number of $n$-cells of $D$.

  In other words, the potential function of $\Delta_{inj} \to Cat$ is morally Euler characteristic.

  As I hinted above, it’s interesting to compare the potential function with what you get if you take the cardinality of the colimit of $B(F-, b)$ instead of its magnitude. I’ll call this the “strict potential function”. In this case, it’s

  $$D \mapsto |colim_{[n] \in \Delta_{inj}} D_n| = |\pi_0(D)|$$

  — the cardinality of the set of connected-components of $D$. So while the potential function gives Euler characteristic, the strict potential function gives the number of connected components.

• Similarly, the potential function of the usual functor $\Delta_{inj} \hookrightarrow Top$, sending $n$ to $\Delta^n$, is given by

  $$D \mapsto \sum_{n \geq 0} (-1)^n |Top(\Delta^n, D)|$$

  for topological spaces $D$. Again, this formally looks like Euler characteristic: the alternating sum of the number of cells in each dimension. And much as for categories, the strict potential function gives the number of path-components.

• For the inclusion $\{primes\} \hookrightarrow (\mathbb{Z}, |)$ that we saw earlier, the potential function $h: \mathbb{Z} \to \mathbb{Q}$ is what’s often denoted by $\omega$, sending an integer $n$ to the number of distinct prime factors of $n$. The strict potential function is the same.

• An easy example that works for any $V$: if $F: A \to B$ has a right adjoint then since $A(F-, b)$ is representable, its magnitude is equal to the cardinality of its colimit, which is $1$. So $h_F: \ob(B) \to k$ has constant value $1$, as does the strict potential function.

  More generally, if $A(F-, b)$ is a coproduct of $n$ representables then $h_F(b) = n$, and the same is true for the strict potential function.

• For the inclusion $F: Field^{op} \hookrightarrow Ring^{op}$, we saw that

  $$N_F(R) = \sum_{p \in Spec(R)} Field(Frac(R/p), -)$$

  for every ring $R$. This is a coproduct of representables, so by the previous example, $|N_F(R)| = |Spec(R)|$. That is, the potential function is given by

  $$h_F: R \mapsto |Spec(R)|$$

  (as is the strict potential function). Usually $Spec(R)$ is infinite, so this calculation is dodgy, but if we restrict ourselves to finite fields and rings then everything is above board.

• For an opfibration $F: A \to B$, one can show that the potential function $h_F$ is given by

  $$h_F(b) = |F^{-1}(b)|.$$

  I should explain the notation on the right-hand side. The category $F^{-1}(b)$ is the fibre over $b \in B$, consisting of the objects of $A$ that $F$ sends to $b$ and the maps in $A$ that $F$ sends to $1_b$. We can take its magnitude (at least if it’s finite and lightning doesn’t strike), which is what I’ve denoted by $|F^{-1}(b)|$.

  I won’t include the proof of this result, but I want to emphasize that it doesn’t involve finding a coweighting on $A$. You might think it would, because the definition of $h_F(b)$ involves the coweighting on $A$. But it turns out that it’s enough to just assume it exists.

• Now take $V = \mathbb{R}^+$, so that $V$-categories are generalized metric spaces. The potential function of a map $f: A \to B$ of metric spaces is the function $h_f : B \to \mathbb{R}$ given by

  $$h_f(b) = \sum_{a \in A} w^A(a) e^{-d(f(a), b)}.$$

  In particular, if $f$ is the inclusion of a subspace $A \subseteq B$, then it’s natural to write $h_A$ instead of $h_f$, and we have

  $$h_A(b) = \sum_{a \in A} w^A(a) e^{-d(a, b)}.$$

  This is equal to $1$ on $A$ but can take other values on $B \setminus A$. In suitable infinite contexts, sums become integrals and $w^A$ becomes something like a measure or distribution, in which case the formula becomes

  $$h_A(b) = \int_A e^{-d(a, b)} \, d w^A(a).$$

  This is exactly the formula for the potential function in Part 1, with one difference: there, I used weightings on $A$, and here, I’m using coweightings. It’s coweightings that one should use. In the previous post, I assumed that all metrics were symmetric, which means that weightings and coweightings are the same. So there’s no inconsistency.

  (Of course, one could take duals throughout and use weightings on $A$ instead. But we’ll see that whichever choice you make, you end up having to consider weightings on one of $A$ and $B$ and coweightings on the other.)

  What about the strict potential function, sending a point $b \in B$ to the “cardinality of the colimit of $B(f-, b): A \to \mathbb{R}^+$”? Well, I put that phrase in inverted commas because we’re now in an enriched context, so it needs a bit of interpretation. “Cardinality” is okay: it becomes the size function $|\cdot| = e^{-(\cdot)}: \mathbb{R}^+ \to \mathbb{R}$. “Colimit” wouldn’t usually make sense in an enriched world, but we’re saved by the fact that the monoidal unit of $V = \mathbb{R}^+$ (namely, $0$) is terminal. The colimit of a $V$-functor into $V$ is just its infimum. So the strict potential function of a map $f: A \to B$ of metric spaces is just

  $$b \mapsto e^{-\inf_{a \in A} d(f(a), b)} = e^{-d(f A, B)}.$$

  I showed you an example of the difference between the potential function and the strict potential function in Part 1, although not with those words. If we take $f$ to be the inclusion $\{-1, 1\} \hookrightarrow \mathbb{R}$ then the potential function is

  

  whereas the strict potential function is

  

  These two functions are identical on $(-\infty, -1] \cup [1, \infty)$, but different on the interval $(-1, 1)$. If you’ve ever wondered what the difference is between strict and lax colimits, here’s an example!

By definition, the potential function of an enriched functor $F: A \to B$ is given by

$$h_F(b) = \sum_{a \in A} w^A(a) |B(F(a), b)|,$$

but a slightly different viewpoint is sometimes helpful. We can take the function $w^A: ob(A) \to k$ and push it forward forward along $F$ to obtain a new function

$$\begin{array}{cccc} F_* w^A: &ob(B) &\to &k, \\ &b &\mapsto &\sum_{a \in F^{-1}(b)} w^A(a). \end{array}$$

Really it’s best to think of $w^A$ as not a function but a measure taking values in $k$ (although these are the same thing on a finite set). Then this is just the usual pushforward measure construction. In any case, the formula for the potential function now becomes

$$h_F(b) = \sum_{b' \in B} (F_* w^A)(b') |B(b', b)|,$$

which has the advantage that everything is taking place in $B$ rather than $A$. In the case where $f: A \to B$ is an embedding of a subspace of a metric space, $f_* w^A$ is just the extension of the measure $w^A$ on $A$ to all of $B$, which we’d usually just write as $w^A$ (with a wink to the audience). In integral notation, the last formula becomes

$$h_F(b) = \int_B e^{-d(x, b)} \, d w^A(x),$$

which is what we had in the last post.

I’ve now explained what potential functions are. But what are they good for?

Last time, I explained that they’re very good indeed for helping us to calculate the magnitude of metric spaces. The key was that

$$|A| = \int_B h_A \, d w_B$$

for a subspace $A$ of a metric space $B$. And as I recounted, that key unlocks the door to the world of PDE methods.

So you might hope something similar is true for enriched categories in general: that there’s a formula for the magnitude in terms of the potential function. And there is! For any $V$-functor $F: A \to B$, it’s a theorem that

$$|A| = \sum_{b \in B} h_F(b) w_B(b).$$

This is an easy calculation from the definitions.

(If you’re really paying attention, you’ll notice that we used the coweighting on $A$ to define the potential function, and now we’re using the weighting on $B$. That’s just how it turns out. One is a weighting, and the other is a coweighting.)

Examples

• For a $V$-functor $F: A \to B$ that has a right adjoint, we saw that the potential function $h_F$ has constant value $1$, so this formula tells us that

  $$|A| = \sum_{b \in B} w_B(b).$$

  But the right-hand side is by definition the magnitude of $B$, so what this formula is saying is that

  $$|A| = |B|.$$

  In other words, if there’s an adjunction between two categories, their magnitudes are equal!

  This has been known forever (Proposition 2.4), and is also intuitive from a homotopical viewpoint. But it’s nice that it just pops out.

• Less trivially, we saw above that for an opfibration $F: A \to B$, the potential function is $h_F: b \mapsto |F^{-1}(b)|$. So

  $$|A| = \sum_{b \in B} w_B(b) |F^{-1}(b)|.$$

  In other words, the magnitude of the “total” category $A$ is the weighted sum over the base $B$ of the magnitudes of the fibres. (Special case: the magnitude of a product is the product of the magnitudes.)

  Now this has been known forever too (Proposition 2.8), but I want to emphasize that the proof is fundamentally different from the one I just linked. That proof constructs the weighting on $A$ from the weightings on the base $B$ and the fibres. Now that’s an easy and informative proof, but what we’ve just done is different, because it didn’t involve figuring out the weighting or coweighting on $A$. So although the result isn’t new or difficult, it’s perhaps grounds for optimism that the method of potential functions will let us prove new things about enriched categories other than metric spaces.

What new things? I don’t know! This is where I’ve got to now. Maybe there are applications in the metric world in which $f: A \to B$ is nothing like an inclusion. Maybe there are applications to graphs, replacing the PDE methods used for subspaces of $\mathbb{R}^n$ by discrete analogues. Maybe the potential function method can be used to shed light on the tricky result that the magnitude of graphs is invariant under certain Whitney twists (Theorem 5.2), and more generally under the sycamore twists introduced by Emily Roff (Theorem 6.5). Let’s find out!

I posted this on my Facebook, but several friends asked me to share more widely, so here goes:

I voted against Trump three times, and donated thousands to his opponents. I’d still vote against him today, seeing him as a once-in-a-lifetime threat to American democracy and even to the Enlightenment itself.

But last night I was also grateful to him for overruling the isolationists and even open antisemites in his orbit, striking a blow against the most evil regime on the planet, and making it harder for that regime to build nuclear weapons. I acknowledge that his opponents, who I voted for, would’ve probably settled for a deal that would’ve resulted in Iran eventually getting nuclear weapons, and at any rate getting a flow of money to redirect to Hamas, Hezbollah, and the Houthis.

May last night’s events lead to the downfall of the murderous ayatollah regime altogether, and to the liberation of the Iranian people from 46 years of oppression. To my many, many Iranian friends: I hope all your loved ones stay safe, and I hope your great people soon sees better days. I say this as someone whose wife and 8-year-old son are right now in Tel Aviv, sheltering every night from Iranian missiles.

Fundamentally, I believe not only that evil exists in the world, but that it’s important to calibrate evil on a logarithmic scale. Trump (as I’ve written on this blog for a decade) terrifies me, infuriates me, and embarrasses me, and through his evisceration of American science and universities, has made my life noticeably worse. On the other hand, he won’t hang me from a crane for apostasy, nor will he send a ballistic missile to kill my wife and son and then praise God for delivering them into his hands.

Update: I received the following comment on this post, which filled me with hope, and demonstrated more moral courage than perhaps every other anonymous comment in this blog’s 20-year history combined. To this commenter and their friends and family, I wish safety and eventually, liberation from tyranny.

I will keep my name private for clear reasons. Thank you for your concern for Iranians’ safety and for wishing the mullah regime’s swift collapse. I have fled Tehran and I’m physically safe but mentally, I’m devastated by the war and the internet blackout (the pretext is that Israeli drones are using our internet). Speaking of what the mullahs have done, especially outrageous was the attack on the Weizmann Institute. I hope your wife and son remain safe from the missiles of the regime whose thugs have chased me and my friends in the streets and imprisoned my friends for simple dissent. All’s well that ends well, and I hope this all ends well.

Dimensionality reduction (the basic being PCA) is very sensitive to outliers: A single bad pixel can dominate most objectives and thus create a spurious dimension. One of the best and most classic solutions to this is the robust PCA method, which is presented in a (very long) paper with impressive math and beautiful results. Yesterday Hans-Walter Rix (MPIA) and I coded it up and applied it to ESA Gaia RVS spectra, with extensive (and impressive) help from Claude. It looks very promising, especially in capturing oddities in hot stars. Today I worked out that there should be something similar that takes into account data weights (inverses of squared uncertainties), and I wrote down the algorithm (on paper). We'll see.

I saw a very nice talk today by Philippine Griveaud (MPIA) about how the Solar System formed. The idea is that the giant planets formed in an accretion disk. Their formation opened gaps and caused migration (first Type I and then Type II, if you must know :). That migration pulled them into a resonant chain. That is, if the giant planets formed the way we think they formed, they must have been in a resonant chain. But they aren't in such a chain now; what gives?

The idea is that when the gas is expended (or blown out by winds), the remaining planetestimals (think: asteroids, comets, Kuiper Belt objects) interact with the planets such that they get moved from orbit to orbit and eventually ejected. These dynamical interactions break the resonant chain, migrate the giant planets to their current locations, and scatter rocks and ice balls into the interstellar regions.

It was a great talk, but also led to a lot of interesting questions, such as: How does this all fit in with the formation of the rocky planets? And how does this square with our observations (growing rapidly, apparently) of interstellar asteroids? Oh and: How does all this connect to observations of debris disks, which I now (officially) love.

In the Milky Way meeting at MPIA today, a bit of a discussion broke out about using stellar twins, inspired by work by Yuan-Sen Ting (OSU). The idea is: If you have two stars with very similar overall metallicity, and very similar temperature and surface gravity, then it should be possible to measure accurate element abundnace anomalies between the two stars, even in the absence of an extremely accurate spectral synthesis code.

My view, which does not contradict this point, is that an even better way to use this stellar-twin idea is to synthesize a twin for every star, using stars that are similar in (either) parameters or else spectral space. After all, an interpolation to your target star should more accurately represent it than even the most similar individual comparison star. That idea, fundamentally, is the main idea behind The Cannon.

guest post by Nathaniel D. Osgood

Together with 4 students from our Computational Epidemiology and Public Health Informatics Lab (CEPHIL), I spent my Friday at one of our community group model building event, this one focused on drivers for homelessness in our city (Saskatoon, Canada).

Although our behavioural ethics review board stipulated that the group should not include people who are currently homeless, the participants were people with lived experience of homelessness, with most having personally experienced homelessness within recent years.

Building on facilitated discussion, the focus of the day consisted of a group model building session. To allow for some diversity of thought and exploration, the participants divided into two teams. The causal loop diagrams resulting from about 1.5-2 hours of work on the part of each team were very thoughtful, and the diagrams captured many important insights and perspectives, and lived experiences. It bears emphasis that while some of those from our lab helped facilitate the discussions, the identification of variables and the existence, directionality and polarity of the links between such variables came firmly from the participants with lived experience themselves. Although they are not yet suitable for public distribution, I thought that I would provide a glimpse of the work products.

For the next stages of this work, CatColab will be a key tool—and arguably the single most important tool in our toolbox to secure substantive insights and value from these diagrams. Building on the strong applied category theory experience of 3 of the 4 students involved, we will be using CatColab to find feedback loops in these diagrams individually, and then when combined. This ability to find feedbacks in this fashion will be a tremendous asset for learning from these diagrams. It will also be used to visualize and explore the diagrams, although it will not be the only tool to serve in this capacity. When the ability to compose causal loop diagrams is added to CatColab, we plan to make central use of that feature as well.

Friday’s event is the first in a series focusing on this pressing problem in our community through tapping the deep and grounded knowledge of those with lived experience. It is a great testimonial to the power of CatColab that it will play such a central role in the value delivery from such events. We hope to contribute to the development of CatColab to further its ability to deliver insights and benefits not only to our research team, but also to community members themselves. I wish to express my—indeed, our—gratitude to the core CatColab team for their delivery of such a valuable tool for insight into complex social issues such as homelessness (together with cognate issues such as mental health, domestic violence and substance use, lack of affordable housing), and CEPHIL’s committment to contributing to the development of that tool to further develop its potential in this sphere.

The Salem prize was established in 1968 and named in honor of Raphaël Salem (1898-1963), a mathematician famous notably for his deep study of the links between Fourier series and number theory and for pioneering applications of probabilistic methods to these fields. It was not awarded from 2019-2022, due to both the COVID pandemic and the death of Jean Bourgain who had been almost single-handedly administering the prize, but is now active again, being administered by Akshay Ventakesh and the IAS. I chair the scientific committee for this prize, whose other members are Guy David and Mikhail Sodin. Last year, the prize was awarded to Miguel Walsh and Yilin Wang.

Nominations for the 2025 Salem Prize are now open until September 15th. Nominations should include a CV of the nominee and a nomination letter explaining the significance of the nominee’s work. Supplementary documentation, such as supporting letters of recommendation or key publications, can additionally be provided, but are not required.

Nominees may be individuals from any country or institution. Preference will be given to nominees who have received their PhD in the last ten years, although this rule may be relaxed if there are mitigating personal circumstances, or if there have been few Salem prize winners in recent years.  Self-nominations will not be considered, nor are past Prize winners or Scientific Committee members eligible.

The prize does not come with a direct monetary award, but winners will be invited to visit the IAS and to give a lecture associated with the award of the prize.

See also the previous year’s announcement of the Salem prize nomination period.

As many of you are no doubt aware, in the past few days the US Congress voted to make major cuts to scientific research, and the president signed the bill. The government’s National Science Foundation has been cut by more than half, which means that its actual science budget has been cut by much more than that after you account for fixed costs. So vast, sudden and draconian are these cuts that it will take a long time for me and others in the field to figure out what has actually happened.

The reductions seem extreme, quite arbitrary and very poorly thought out. As an example, half of the LIGO observatory (the Laser Interferometer Gravitational-Wave Observatory, whose amazing discoveries, such as this one and this one, earned the United States a Nobel Prize in 2017) is being hit hard. There are currently two interferometers, one in Washington state and one in Lousiana, but one has been largely defunded in this bill, if I understand correctly.

I can see the logic: the scientists have two interferometers, but in tough times they ought to be able to get along with just one, right?

Well, that’s like cutting off one of a runner’s legs. Two were built because two were needed.

With just one, the signal from most gravitational wave events is so weak that you can’t distinguish it from noise. Other interferometers around the world just aren’t working well enough to make up for throwing away one of LIGOs. (And besides, you need three or four interferometers around the world to be able to know precisely in the sky where the waves are coming from, knowledge which can make other major discoveries possible.)

According to Science magazine, “In a two-sentence email to Science, an NSF spokesperson said the plan reflects `a strategic alignment of resources in a constrained fiscal environment.’ “

This is not strategic. This is stupid. The amount of money saved, less than 10 cents per year per US citizen, is very small compared to what we as a nation have already spent on this wonderful facility, and cutting LIGO in half makes it dramatically less than half as good — so this is actually a big waste of money both past and future. The decision to make this cut in this way is nothing short of ridiculous and incompetent.

[Not to mention that “constrained fiscal environment” is quite a phrase when you’re increasing the budget deficit rather than shrinking it.]

I fear there are many other similar examples to be found.

David Suzuki is an 89-year-old Canadian geneticist, science broadcaster and environmental activist. In this interview he says some things that I’ve come to agree with.

• ‘It’s too late’: David Suzuki says the fight against climate change is lost, iPolitics, 2 July 2025.

Q: It’s clear you haven’t lost your passion for a lot of the issues that you care about, but do you ever feel like you’re banging your head against the wall? If you look at public opinion data, climate change is often well down the list of priorities for most Canadians.

When you see that, where do you find the motivation to continue speaking to the values you believe are important?

A: I believe an informed public will do the right thing. Public concern in the late 1980s was right at the top and we had the first international conference on the atmosphere in 1988, where there were 300 people, over 40 governments, environmentalists, scientists, private sector people, you name it.

At the end of that conference, they said global warming represented a threat to humanity, second only to global nuclear war. If the world had followed the conclusions from that conference, we would not have the problem we face today and we would have saved trillions of dollars and millions of lives.

Now, it is too late.

I’ve never said this before to the media, but it’s too late. I say that because I go by science and Johan Rockström, the Swedish scientist who heads the Potsdam Institute, has defined nine planetary boundaries. These are constraints on how we live. As long as humans, like any other animal, live within those nine constraints, we can do it forever, and that includes the amount of carbon in the atmosphere, the pH of the oceans, the amount of available fresh water, the nitrogen cycle, etc.

There are nine planetary boundaries and we’ve only dealt with one of them—the ozone layer—and we think we’ve saved ourselves from that threat. But we passed the seventh boundary this year, and we’re in the extreme danger zone. Rockström says we have five years to get out of the danger zone.

If we pass one boundary, we should be shitting our pants. We’ve passed seven!

And, if you look at those boundaries, like the amount of carbon in the atmosphere, we’ve had 28 COP meetings on climate change and we haven’t been able to cap emissions.

We’re on our way to more than a three-degree temperature rise by the end of this century, and scientists agree we shouldn’t rise above one and half degrees.

Q: You say we’re too late to address climate change? That’s a pretty stark quote. Does that mean you’re giving up on the fight?

A: I’m not giving up on the immediate years, but the focus on politics, economics, and law are all destined to fail because they are based around humans. They’re designed to guide humans, but we’ve left out the foundation of our existence, which is nature, clean air, pure water, rich soil, food, and sunlight. That’s the foundation of the way we live and, when we construct legal, economic and political systems, they have to be built around protecting those very things, but they’re not.

Q: You’ve been quoted many times over the last couple decades saying it’s not too late to tackle climate change, so when did you come to this realization that the battle is lost?

A: It’s been coming all along.

We had previously said that the choice with climate change was mitigation and adaptation, and people began saying 20 years ago that we had to talk about adaptation. Other people said we can’t talk about adaptation because that acknowledges that climate change is real and impacting people. Well, we’re way past the time when we should have been thinking about adaptation.

Look, I’m not giving up in the sense of not doing anything, but Trump’s election was the dagger in my heart. Trump’s win was the triumph of capitalism and neoliberalism, and he’s going to wreak havoc. There’s nothing we can do about that, except maybe incremental changes. That’s not what we need. We need revolution. Can you have a peaceful revolution? I don’t know.

But I’m saying, as an environmentalist, we have failed to shift the narrative and we are still caught up in the same legal, economic and political systems.

For me, what we’ve got to do now is hunker down. The units of survival are going to be local communities, so I’m urging local communities to get together. Finland is offering a great example because the Finnish government has sent a letter to all of their citizens warning of future emergencies, whether they’re earthquakes, floods, droughts, or storms. They’re going to come and they’re going to be more urgent and prolonged.

Governments will not be able to respond on the scale or speed that is needed for these emergencies, so Finland is telling their citizens that they’re going to be at the front line of whatever hits and better be sure you’re ready to meet it. Find out who on your block can’t walk because you’re going to have to deal with that. Who has wheelchairs? Who has fire extinguishers? Where is the available water? Do you have batteries or generators? Start assessing the roots of escape. You’re going to have to inventory your community, and that’s really what we have to start doing now.

It’s hard to know what to do with my life if these things are true, because my life was based on other assumptions. I also believe there’s a decent chance I can eke out a comfortable existence for a couple more decades without dramatically changing my behavior (apart from moving to Scotland). But it would be silly not to update my assumptions, state this publicly, and take it from there.

It sounds like Suzuki, too, was reluctant to say publicly what he’s now come to believe. But I’m glad he’s come out and said it. If the basic outlines are correct, there’s a lot to do, and no point in staying quiet about it, even if we don’t yet have the courage and energy to change our lives as the situation calls for.

read more

Good advice I got from a colleague. If you have advisees who don’t have US citizenship, as many of us do, tell them directly that you are their advisor, no matter what, wherever they are located, and that no governmental entity has authority over that relationship.

Three of my science journalism pieces went up last week!

(This is a total coincidence. One piece was a general explainer “held in reserve” for a nice slot in the schedule, one was a piece I drafted in February, while the third I worked on in May. In journalism, things take as long as they take.)

The shortest piece, at Quanta Magazine, was an explainer about the two types of particles in physics: bosons, and fermions.

I don’t have a ton of bonus info here, because of how tidy the topic is, so just two quick observations.

First, I have the vague impression that Bose, bosons’ namesake, is “claimed” by both modern-day Bangladesh and India. I had friends in grad school who were proud of their fellow physicist from Bangladesh, but while he did his most famous work in Dhaka, he was born and died in Calcutta. Since both were under British India for most of his life, these things likely get complicated.

Second, at the end of the piece I mention a “world on a wire” where fermions and bosons are the same. One example of such a “wire” is a string, like in string theory. One thing all young string theorists learn is “bosonization”: the idea that, in a 1+1-dimensional world like a string, you can re-write any theory with fermions as a theory with bosons, as well as vice versa. This has important implications for how string theory is set up.

Next, in Ars Technica, I had a piece about how LHC physicists are using machine learning to untangle the implications of quantum interference.

As a journalist, it’s really easy to fall into a trap where you give the main person you interview too much credit: after all, you’re approaching the story from their perspective. I tried to be cautious about this, only to be stymied when literally everyone else I interviewed praised Aishik Ghosh to the skies and credited him with being the core motivating force behind the project. So I shrugged my shoulders and followed suit. My understanding is that he has been appropriately rewarded and will soon be a professor at Georgia Tech.

I didn’t list the inventors of the NSBI method that Ghosh and co. used, but names like Kyle Cranmer and Johann Brehmer tend to get bandied about. It’s a method that was originally explored for a more general goal, trying to characterize what the Standard Model might be missing, while the work I talk about in the piece takes it in a new direction, closer to the typical things the ATLAS collaboration looks for.

I also did not say nearly as much as I was tempted to about how the ATLAS collaboration publishes papers, which was honestly one of the most intriguing parts of the story for me. There is a huge amount of review that goes on inside ATLAS before one of their papers reaches the outside world, way more than there ever is in a journal’s peer review process. This is especially true for “physics papers”, where ATLAS is announcing a new conclusion about the physical world, as ATLAS’s reputation stands on those conclusions being reliable. That means starting with an “internal note” that’s hundreds of pages long (and sometimes over a thousand), an editorial board that manages the editing process, disseminating the paper to the entire collaboration for comment, and getting specific experts and institute groups within the collaboration to read through the paper in detail. The process is a bit less onerous for “technical papers”, which describe a new method, not a new conclusion about the world. Still, it’s cumbersome enough that for those papers, often scientists don’t publish them “within ATLAS” at all, instead releasing them independently. The results I reported on are special because they involved a physics paper and a technical paper, both within the ATLAS collaboration process. Instead of just working with partial or simplified data, they wanted to demonstrate the method on a “full analysis”, with all the computation and human coordination that requires. Normally, ATLAS wouldn’t go through the whole process of publishing a physics paper without basing it on new data, but this was different: the method had the potential to be so powerful that the more precise results would be worth stating as physics results alone.

(Also, for the people in the comments worried about training a model on old data: that’s not what they did. In physics, they don’t try to train a neural network model to predict the results of colliders, such a model wouldn’t tell us anything useful. They run colliders to tell us whether what they see matches the analytic, Standard, model. The neural network is trained to predict not what the experiment will say, but what the Standard Model will say, as we can usually only figure that out through time-consuming simulations. So it’s trained on (new) simulations, not on experimental data.)

Finally, on Friday I had a piece in Physics Today about the European Strategy for Particle Physics (or ESPP), and in particular, plans for the next big collider.

Before I even started working on this piece, I saw a thread by Patrick Koppenburg on some of the 263 documents submitted for the ESPP update. While my piece ended up mostly focused on the big circular collider plan that most of the field is converging on (the future circular collider, or FCC), Koppenburg’s thread was more wide-ranging, meant to illustrate the breadth of ideas under discussion. Some of that discussion is about the LHC’s current plans, like its “high-luminosity” upgrade that will see it gather data at much higher rates up until 2040. Some of it is assessing broader concerns, which it may surprise some of you to learn includes sustainability: yes, there are more or less sustainable ways to build giant colliders.

The most fun part of the discussion, though, concerns all of the other collider proposals.

Some report progress on new technologies. Muon colliders are the most famous of these, but there are other proposals that would specifically help with a linear collider. I never did end up understanding what Cooled Copper Colliders are all about, beyond that they let you get more energy in a smaller machine without super-cooling. If you know about them, chime in in the comments! Meanwhile, plasma wakefield acceleration could accelerate electrons on a wave of plasma. This has the disadvantage that you want to collide electrons and positrons, and if you try to stick a positron in plasma it will happily annihilate with the first electron it meets. So what do you do? You go half-and-half, with the HALHF project: speed up the electron with a plasma wakefield, accelerate the positron normally, and have them meet in the middle.

Others are backup plans, or “budget options”, where CERN could get a bit better measurements on some parameters if they can’t stir up the funding to measure the things they really want. They could put electrons and positrons into the LHC tunnel instead of building a new one, for a weaker machine that could still study the Higgs boson to some extent. They could use a similar experiment to produce Z bosons instead, which could serve as a bridge to a different collider project. Or, they could collider the LHC’s proton beam with an electron beam, for an experiment that mixes advantages and disadvantages of some of the other approaches.

While working on the piece, one resource I found invaluable was this colloquium talk by Tristan du Pree, where he goes through the 263 submissions and digs up a lot of interesting numbers and commentary. Read the slides for quotes from the different national inputs and “solo inputs” with comments from particular senior scientists. I used that talk to get a broad impression of what the community was feeling, and it was interesting how well it was reflected in the people I interviewed. The physicist based in Switzerland felt the most urgency for the FCC plan, while the Dutch sources were more cautious, with other Europeans firmly in the middle.

Going over the FCC report itself, one thing I decided to leave out of the discussion was the cost-benefit analysis. There’s the potential for a cute sound-bite there, “see, the collider is net positive!”, but I’m pretty skeptical of the kind of analysis they’re doing there, even if it is standard practice for government projects. Between the biggest benefits listed being industrial benefits to suppliers and early-career researcher training (is a collider unusually good for either of those things, compared to other ways we spend money?) and the fact that about 10% of the benefit is the science itself (where could one possibly get a number like that?), it feels like whatever reasoning is behind this is probably the kind of thing that makes rigor-minded economists wince. I wasn’t able to track down the full calculation though, so I really don’t know, maybe this makes more sense than it looks.

I think a stronger argument than anything along those lines is a much more basic point, about expertise. Right now, we have a community of people trying to do something that is not merely difficult, but fundamental. This isn’t like sending people to space, where many of the engineering concerns will go away when we can send robots instead. This is fundamental engineering progress in how to manipulate the forces of nature (extremely powerful magnets, high voltages) and process huge streams of data. Pushing those technologies to the limit seems like it’s going to be relevant, almost no matter what we end up doing. That’s still not putting the science first and foremost, but it feels a bit closer to an honest appraisal of what good projects like this do for the world.

Jesper Grimstrup kindly sent me an electronic copy of his new book, The Ant Mill. He was also kind enough to give me some feedback on a first version of this review.

The story is an interweaving of the author's personal experiences in academia with his general observations. This personal story and his experiences are interesting, much like I expect those of everyone who has spent several years in academia would be. And he has clearly spent a lot of time thinking about thinking about research. I love meta-activities of this sort; the best example that I know of is You and Your Research by Hamming, which I stumbled on as a postdoc. Indeed, the existence of these sorts of things that are shared by young researchers is actually evidence against the central thesis of Grimstrup's book.

The market attacking High-Energy Physics seems to be burgeoning. On the one hand Hossenfelder believes that we have become 'lost in math,' and on the other Woit believes we are not mathematical enough; both attack string theory as a failed program. Grimstrup's book is in the mathematical camp, with the novelty that he piles scorn on all popular approaches to quantum gravity, in particular loop quantum gravity and noncommutative geometry, since he has come into closest contact with them. His observations about string theorists are mainly about the shoddy way that he was treated during his time at the NBI, with several egregious examples of bad behaviour. We are lead to conclude that it is not just string theorists who have formed a closed tribe, but that there are several such groups crowding out innovation.

One problem with the combination of the general and the personal is that Grimstrup constantly refers to his own research program and gives examples of how it has just generally been ignored within academia. For example, he starts the book with a copy of a grant application by a 31-year-old Niels Bohr for an entire institute, and contrasts this with a grant application of his that was refused that effectively ended his career within academia (my understanding is that at the NBI in Copenhagen it is common to ask for and obtain grants to pay your own salary and prolong temporary contracts). He writes that he does not do this to compare himself to Niels Bohr, but he does this repeatedly (mostly indirectly) throughout the book -- not in a self-aggrandising way, but in the sense that you can almost feel his frustration coming through the pages that his expectations did not meet reality. It seems like bait at times, inviting anyone who disagrees with the general thesis to attack him personally. Instead, I will have a look at his papers with an open mind, after writing this review, and keep my thoughts on them to myself.

The book made me think of how many of us enter academia. We grow up reading popular science accounts idolising physicists from a century ago. And it made me think more of the self-actualisation messages that were rammed down all our throats in the popular culture in the 80s and 90s: follow your dreams, stick to your principles, be true to yourself, this is the most important thing in life and you shouldn't worry about money, just be happy. And: working hard and getting good grades is the way to get to the top. The problem is that this is largely obsolete: it's based on the world that existed post world war two when there was a scarcity of labour and an economic boom. Then -- if you were from the right background and your face fit -- you could work hard, get a PhD and walk into a permanent academic job (yes this is a caricature). Science was respected and so were scientists; high-energy physics was at the top of the tree because of the connection with technological advancements and nuclear weapons. That world doesn't exist any more; while in many ways for the better, it is undeniable that we live in a world of much greater competition and public skepticism about science is increasing.

The scientific community has expanded, as has the population; and more importantly education throughout the world and global travel and communication has meant that the number of people around the world who are involved in research is much greater than it was. Grimstrup notes that increasing the size of the academic community has led to fundamental changes of behaviour: professionalisation of research and group think. It is clearly true that the Matthew effect exists in many branches of society, and therefore also in academia; governments wanting to exert some form of oversight in exchange for the funds that they provide has definitely led to changes in incentives for researchers. One aspect of this is that it is hard to judge the work of people from other fields, but we are required to do so; and then it is difficult to argue with quantitative measures such as number of papers, citations, h-indices. Then of course the measure becomes the target for certain people. 

Grimstrup rails against all these changes; he clearly believed that the correct thing to do for an aspiring researcher would be to work on their own ideas, stick to their principles and not compromise. They should work for a long time, in isolation, on a major paper, put it on arxiv.org and the next day their colleagues would read it and ask interesting questions about them. Fame and fortune would follow. The thing that shocked Grimstrup was that not only did people not even care about any papers he posted, a young competitor even once told him some ideas are simply not worth pursuing even though they may be interesting. For sure, this is horrible and shocking behaviour, and does not reflect well on the anonymous person who said it.

For my part I am still naive enough to think that if new ideas are good, someone will recognise them as such, and network effects will make them known. I know that many researchers already think more deeply about what they are doing than he gives us credit for: and we discuss it, during seminars, over a drink with colleagues, in the coffee-breaks of conferences, during our annual or five-year reviews, or in grant applications. When I discussed this review with a string-theorist colleague they remarked "of course we know the situation sucks!''  I think Grimstrup is therefore wrong to tar everyone with the same brush: the diversity in our community has increased greatly with time, and this means that there are indeed strong incentives to take a risk on a novel idea, because the rewards of opening a new research direction are immense! Being the originator of an idea, or the first to recognise the merit in even an old forgotten idea, can yield tremendous results and even greater recognition nowadays thanks to the same effects. Hence, starting a new field, or even a subfield, is something that most researchers aspire to; the rewards for doing so are even greater now than in times gone by, and the evidence that this is possible is even given in this book: the existence of several communities working on different approaches to quantum gravity. He argues that these are now old and stale, but my point is that the way that they were able to take root at all is an example of how this can happen. There are many subfields that have sprung up more recently, and in other branches of HEP there are of course many examples. Nowadays things can change very quickly: a new good idea will be very rapidly jumped on once it is recognised, and people are constantly on the lookout. 

Grimstrup also, like Lee Smolin, divides researchers into visionaries and technicians. He then complains that the technicians have taken over, with lots of disparaging comments about them digging endless holes. He then complains that there is an incentive to collaborate in modern research, only collaborators survive in the system: he has evidence that being a lone wolf is a poor survival strategy. He believes that we should work on our own; yet at the same time visionaries need to collaborate with technicians. I found this very jarring. Other than the facile placing of people into boxes, he is overlooking the benefits of collaboration -- his opinion is that it is just about inflating the number of papers one person can sign (and for sure there are people who cynically do this). But to me, discussing with other people, even just explaining something, is often the quickest way to generate genuinely new ideas or solutions to problems that we may never have come up with alone. At the same time, there are plenty of people who do write papers alone; to take a leaf from his book and share a personal story, I once had a comment on a postdoc application that I had no single-author papers and therefore did not demonstrate independence. Hence, there are incentives and a good reason for young researches to work alone sometimes. I then wrote a single-author paper, as I have occasionally done since (and got the fellowship next time I applied); I would agree that there is a pleasure and some advantages in doing this, but to do this all the time would mean I would risk missing out on lots of new ideas and other perspectives, as well as the pleasure of regular interactions with collaborators, and it would also limit the scope of my projects, where I benefit from others' expertise. Or collaborations may just be working with a student, pursuing my ideas (hopefully they contribute some of their own!) and imparting my knowledge in the process. This is why I do not think that encouraging people to predominantly cloister themselves away to work alone for a long time is the most productive or healthy one. 

The book also has a very narrow focus as to the goal of high-energy physics. For the author, the quest is a "the next theory," but in essence this means a theory of quantum gravity, which he acknowledges would be far from being able to be tested with any present or near-future data. Otherwise, we should look for a mathematically rigorous definition of quantum field theory; he hopes these will be one and the same thing. This latter problem has proven to be both very hard and not obviously useful -- it is certainly not obvious that the solution should even be unique, for example a theory of strings would cure ultra-violet divergences, and the question of whether strings should be necessary for such a theory is one that I know people have tried to explore. I also recently attended a talk by Michael Douglas where he reviewed recent attempts on rigorous QFT, so it is a subject that is regarded as important but very difficult, and still being explored by a small number of people. Regarding quantum gravity, some people in the community have taken the opinion that if you have no data, it is not a good problem, and are working on other things. Or people try to make contact with data using e.g. EFT approaches to measuring quantum effects of gravity. The string theory community might say that we do have a theory of quantum gravity, in fact we have a landscape of them, and try e.g. to use it to answer questions about black hole information. But at the same time some people then complain that the leading string theorists have moved on to other things: there are lots of important open fundamental problems, and we just do not know how they are interlinked, if at all!

Grimstrup's insistence that the solution to what he sees as problems is to shrink competition and also encourage research outside of academia, reminded me of another Dane, subject of another book I read recently: king Cnut, famous for (presumably apocryphally) standing on the beach in front of his ministers and commanding the tide to turn back. Otherwise Grimstrup hopes for a crisis, perhaps one provoked by his book. He explicitly states that he does not want to fuel the anti-establishment or ant-academic movements, but I suspect that the only crises we might suffer would not be good for the field.  Perhaps one is already taking place in the US; perhaps people will take his message to heart despite his protests and start a DOGE-style decimation of research. Necessarily, in science we mark our own homework: only other scientists are capable of judging the claims of their peers. If we start opening this up to question then we will only end with government appointees deciding what are acceptable topics and directions, or shutting public funding down altogether. What would be left over would surely be even greater competition for scarce resources.

For me, the solution to the problems in the book, to the extent that I agree with them, is to regularly remind ourselves that we should always maintain a childlike curiosity and not close our minds to new ideas and new possibilities. This is the message from the text of Hamming, and very well put in the writings of Feynman (who Grimstrup bizarrely dismisses as a technician compared to Bohr). Otherwise of course in science it is necessary to have a community spirit, to realise that we are all trying to make progress in the best way we know how, and to help each other do so; and it is necessary to maintain healthy competition as a motivator. But both conflicting instincts -- to compete and to group into communities -- are vital parts of human nature and denying this has been the mistake of utopians throughout history. 

I am also sure that many of the complaints that Grimstrup assigns to high-energy physics could also be applied to society more generally. So instead of trying to hold back or reverse the societal changes of the last century we should try to work with them as best we can. We have to accept that we live now in an attention economy; and this gives new opportunities: blogging, social media, writing articles in science magazines or popular press, etc. Since Grimstrup is now, interestingly, an independent scientist, perhaps tying his own research program so closely with his book is embracing the modern world at last, and creating a brand as a radical outside thinker, that will be attractive to private backers. He promotes the path that he has followed, crowdfunding his research or seeking support of patrons, as a possible path for the independently minded once they have completed their training in academia, and in this I wish him well: he is clearly serious, determined and sincere. But while this is now part of twenty-first century society, many people have noticed that this modern trend is a return to the nineteenth century (or even earlier, e.g. Leonardo da Vinci being invited to France by François 1) where a wealthy patron was the only source of funding. 

In work on Galaxy dynamics, from stellar kinematics, we measure relative velocities and relative positions, of nearby stars relative to the Sun (or really the Solar System barycenter). These relative positions and velocities are coordinate free, in the sense that they don't imply a rest frame for anything (and indeed, the SS barycenter is not anywhere near the rest-frame position or rest-frame velocity of the Milky Way or Local Group or anything else).

In addition to this, any measurements we make are insensitive to any overall or external acceleration: If the Milky way is in free-fall, accelerating towards some external “great attractor” or anything else, none of these observables are affected in any way by that acceleration. So what is it that stellar kinematics can really be used to measure? I think somehow the answer has to be Galilean covariant (covariant to boosts and translations), but even better it should be generally covariant (in the Newtonian sense, which is well defined, apparently).

I did some research on this subject, and the literature is all about Newton–Cartan theory, but this theory is a Newtonian limit of general relativity. That isn't quite what we care about in stellar kinematics, since in stellar kinematics, we don't get to see any orbits as a function of time (we don't observe geodesics or geodesic deviation). What, exactly do we observe? I think what we observe is something about gradients of accelerations, but I don't know yet. Great project for this summer.

The basis for much of modern electronics is a set of silicon technologies called CMOS, which stands for complementary metal oxide semiconductor devices and processes.  "Complementary" means using semiconductors (typically silicon) that is locally chemically doped so that you can have both n-type (carriers are negatively charged electrons in the conduction band) and p-type (carriers are positively charged holes in the valence band) material on the same substrate.  With field-effect transistors (using oxide gate dielectrics), you can make very compact, comparatively low power devices like inverters and logic gates.  

There are multiple different approaches to try to implement quantum information processing in solid state platforms, with the idea that the scaling lessons of microelectronics (in terms of device density and reliability) can be applied.  I think that essentially all of these avenues require cryogenic operating conditions; all superconducting qubits need ultracold conditions for both superconductivity and to minimize extraneous quasiparticles and other decoherence sources.  Semiconductor-based quantum dots (Intel's favorite) similarly need thermal perturbations and decoherence to be minimized.  The wealth of solid state quantum computing research is the driver for the historically enormous (to me, anyway) growth of dilution refrigerator manufacturing (see my last point here).

So you eventually want to have thousands of error-corrected logical qubits at sub-Kelvin temperatures, which may involve millions of physical qubits at sub-Kelvin temperatures, all of which need to be controlled.  Despite the absolute experimental fearlessness of people like John Martinis, you are not going to get this to work by running a million wires from room temperature into your dil fridge.  

Fig. 1 from here.

In this most recent work, the aspect that I find most impressive is that the CMOS electronics are essentially a serious logic-based control board operating at milliKelvin temperatures right next to the chip with the qubits (in this case, spins-in-quantum-dots).  I'm rather blown away that this works and with sufficiently low power dissipation that the fridge is happy.  This is very impressive, and there is likely a very serious future in store for cryogenic CMOS.

I’ve been working with Adittya Chaudhuri on some ideas related to this series of blog articles, and now our paper is done!

• John Baez and Adittya Chaudhuri, Graphs with polarities.

Abstract. In fields ranging from business to systems biology, directed graphs with edges labeled by signs are used to model systems in a simple way: the nodes represent entities of some sort, and an edge indicates that one entity directly affects another either positively or negatively. Multiplying the signs along a directed path of edges lets us determine indirect positive or negative effects, and if the path is a loop we call this a positive or negative feedback loop. Here we generalize this to graphs with edges labeled by a monoid, whose elements represent ‘polarities’ possibly more general than simply ‘positive’ or ‘negative’. We study three notions of morphism between graphs with labeled edges, each with its own distinctive application: to refine a simple graph into a complicated one, to transform a complicated graph into a simple one, and to find recurring patterns called ‘motifs’. We construct three corresponding symmetric monoidal double categories of ‘open’ graphs. We study feedback loops using a generalization of the homology of a graph to homology with coefficients in a commutative monoid. In particular, we describe the emergence of new feedback loops when we compose open graphs using a variant of the Mayer–Vietoris exact sequence for homology with coefficients in a commutative monoid.

Read the whole series:

• Part 1: Causal loop diagrams, and more generally graphs with edge labeled by elements of a monoid.

• Part 2: graphs with edges labeled by elements of a ring.

• Part 3: hyperrings and hyperfields.

• Part 4: rigs from hyperrings.

• Part 5: pulling back and pushing forwards edge labels on labeled graphs.

• Part 6: a paper called ‘Graphs with polarities’ with Adittya Chaudhuri, summarizing some of the work here but also much more.

I participated in a program about 15 years ago that looked at science and technology challenges faced by a subset of the US government. I came away thinking that such problems fall into three broad categories.

1. Actual science and engineering challenges, which require foundational research and creativity to solve.
2. Technology that may be fervently desired but is incompatible with the laws of nature, economic reality, or both. 
3. Alleged science and engineering problems that are really human/sociology issues.

Part of science and engineering education and training is giving people the skills to recognize which problems belong to which categories.  Confusing these can strongly shape the perception of whether science and engineering research is making progress. 

There has been a lot of discussion in the last few years about whether scientific progress (however that is measured) has slowed down or stagnated.  For example, see here:

https://www.theatlantic.com/science/archive/2018/11/diminishing-returns-science/575665/ 

https://news.uchicago.edu/scientific-progress-slowing-james-evans

https://www.forbes.com/sites/roberthart/2023/01/04/where-are-all-the-scientific-breakthroughs-forget-ai-nuclear-fusion-and-mrna-vaccines-advances-in-science-and-tech-have-slowed-major-study-says/

https://theweek.com/science/world-losing-scientific-innovation-research

A lot of the recent talk is prompted by this 2023 study, which argues that despite the world having many more researchers than ever before (behold population growth) and more global investment in research, somehow "disruptive" innovations are coming less often, or are fewer and farther between these days.  (Whether this is an accurate assessment is not a simple matter to resolve; more on this below.)

There is a whole tech bro culture that buys into this, however.  For example, see this interview from last week in the New York Times with Peter Thiel, which points out that Thiel has been complaining about this for a decade and a half.  

On some level, I get it emotionally.  The unbounded future spun in a lot of science fiction seems very far away.  Where is my flying car?  Where is my jet pack?  Where is my moon base?  Where are my fusion power plants, my antigravity machine, my tractor beams, my faster-than-light drive?  Why does the world today somehow not seem that different than the world of 1985, while the world of 1985 seems very different than that of 1945?

Some of the folks that buy into this think that science is deeply broken somehow - that we've screwed something up, because we are not getting the future they think we were "promised".  Some of these people have this as an internal justification underpinning the dismantling of the NSF, the NIH, basically a huge swath of the research ecosystem in the US.  These same people would likely say that I am part of the problem, and that I can't be objective about this because the whole research ecosystem as it currently exists is a groupthink self-reinforcing spiral of mediocrity.  

Science and engineering are inherently human ventures, and I think a lot of these concerns have an emotional component.  My take at the moment is this:

1. Genuinely transformational breakthroughs are rare.  They often require a combination of novel insights, previously unavailable technological capabilities, and luck.  They don't come on a schedule.  
2. There is no hard and fast rule that guarantees continuous exponential technological progress.  Indeed, in real life, exponential growth regimes never last. The 19th and 20th centuries were special.   If we think of research as a quest for understanding, it's inherently hierarchal.  Civilizational collapses aside, you can only discover how electricity works once.   You can only discover the germ theory of disease, the nature of the immune system, and vaccination once (though in the US we appear to be trying really hard to test that by forgetting everything).  You can only discover quantum mechanics once, and doing so doesn't imply that there will be an ongoing (infinite?) chain of discoveries of similar magnitude.
3. People are bad at accurately perceiving rare events and their consequences, just like people have a serious problem evaluating risk or telling the difference between correlation and causation.  We can't always recognize breakthroughs when they happen.  Sure, I don't have a flying car.  I do have a device in my pocket that weighs only a few ounces, gives me near-instantaneous access to the sum total of human knowledge, let's me video call people around the world, can monitor aspects of my fitness, and makes it possible for me to watch sweet videos about dogs.  The argument that we don't have transformative, enormously disruptive breakthroughs as often as we used to or as often as we "should" is in my view based quite a bit on perception.
4. Personally, I think we still have a lot more to learn about the natural world.  AI tools will undoubtedly be helpful in making progress in many areas, but I think it is definitely premature to argue that the vast majority of future advances will come from artificial superintelligences and thus we can go ahead and abandon the strategies that got us the remarkable achievements of the last few decades.
5. I think some of the loudest complainers (Thiel, for example) about perceived slowing advancement are software people.  People who come from the software development world don't always appreciate that physical infrastructure and understanding are hard, and that there are not always clever or even brute-force ways to get to an end goal.  Solving foundational problems in molecular biology or quantum information hardware or  photonics or materials is not the same as software development.  (The tech folks generally know this on an intellectual level, but I don't think all of them really understand it in their guts.  That's why so many of them seem to ignore real world physical constraints when talking about AI.).  Trying to apply software development inspired approaches to science and engineering research isn't bad as a component of a many-pronged strategy, but alone it may not give the desired results - as warned in part by this piece in Science this week.  

More frequent breakthroughs in our understanding and capabilities would be wonderful.  I don't think dynamiting the US research ecosystem is the way to get us there, and hoping that we can dismantle everything because AI will somehow herald a new golden age seems premature at best.

For overdetermined reasons, I’ve lately found the world an increasingly terrifying and depressing place. It’s gotten harder and harder to concentrate on research, or even popular science writing. Every so often, though, something breaks through that wakes my inner child, reminds me of why I fell in love with research thirty years ago, and helps me forget about the triumphantly strutting factions working to destroy everything I value.

Back in 2022, I reported an exciting advance in BusyBeaverology: namely, whereas we previously knew merely that BB(6) > 10^36,534, Pavel Kropitz managed to show that

BB(6) > ¹⁵10.

For those tuning in from home, here BB(6) is the 6^th Busy Beaver number, i.e. the maximum number of steps that a 6-state Turing machine with a {0,1} alphabet can take before halting, when run on an initially all-0 input tape. Also, the left-superscript means tetration, or iterated exponentiation: for example, ¹⁵10 means 10 to the 10 to the 10 and so on 15 times.

By comparison, last year the international “BBchallenge” team determined that BB(5) is “merely” 47,176,870 (see also Quanta magazine’s superb feature article on that milestone). So, between 5 and 6 is where the Busy Beaver function makes its leap, from the millions to beyond the bounds of observable reality.

But if you thought that was the end of the BB(6) story, think again! Eleven days ago, Tristan Sterin, who organized the BBchallenge the team, emailed to tell me that a team member with the handle “mxdys” improved the BB(6) bound yet further, to

BB(6) > ^10,000,00010

(i.e., 10 to the 10 to the 10 and so on 10 million times), with a correctness proof in Coq. Then, three days ago, Tristan wrote again to say that mxdys has improved the bound again, to

$$ BB(6) \gt ^{^{{^9}2}2}2 $$

I.e., BB(6) is at least 2 tetrated to the 2 tetrated to the 2 tetrated to the 9. So in particular, BB(6) is at least 2 pentated to the 5, where pentation is iterated tetration, i.e. the operation that is to tetration as tetration is to exponentiation, exponentiation is to multiplication, and multiplication is to addition.

Last week, when we “merely” knew that BB(6) > ^10,000,00010, I talked to a journalist who asked me to give an intuitive sense of how big such a number is. So I said, imagine you had ^10,000,00010 grains of sand. Then you could … well, uh … you could fill about ^10,000,00010 copies of the observable universe with that sand. I hope that helps people visualize it!

The journalist also asked: have these new discoveries about BB(6) caused me to rethink any broader beliefs about the Busy Beaver function? And I mean, yes and no: it was always completely within the realm of possibility that BB(6) would already be, not some puny little thing like 10^36,534, but way out in iteration land. Now that we know for sure that it is, though, maybe I ought to conjecture that the value of BB(n) becomes independent of the ZFC axioms of set theory already when n is 7 or 8 or 9, rather than when it’s 20 or 30 or whatever. (Currently, we know that BB(n) becomes independent of ZFC only when n=643.)

Unrelated Update: I’m just now returning to the US from STOC’2025 in Prague, where I saw lots of old friends and learned many interesting new things, again helping to distract me from the state of the world! Many I’ll write about some of those things in a future post. For now, though, anyone who’s interested in my STOC plenary lecture, entitled “The Status of Quantum Speedups,” can check out the PowerPoint slides here.

You may have heard that the muon g-2 problem has been solved.

Muons are electrons’ heavier cousins. As spinning charged particles, they are magnetic, the strength of that magnetism characterized by a number denoted “g”. If you were to guess this number from classical physics alone, you’d conclude it should be 2, but quantum mechanics tweaks it. The leftover part, “g-2”, can be measured, and predicted, with extraordinary precision, which ought to make it an ideal test: if our current understanding of the particle physics, called the Standard Model, is subtly wrong, the difference might be noticeable there.

And for a while, it looked like such a difference was indeed noticeable. Extremely precise experiments over the last thirty years have consistently found a number slightly different from the extremely precise calculations, different enough that it seemed quite unlikely to be due to chance.

Now, the headlines are singing a different tune.

What changed?

That headline might make you think the change was an experimental result, a new measurement that changed the story. It wasn’t, though. There is a new, more precise measurement, but it agrees with the old measurements.

So the change has to be in the calculations, right? They did a new calculation, corrected a mistake or just pushed up their precision, and found that the Standard Model matches the experiment after all?

…sort of, but again, not really. The group of theoretical physicists associated with the experiment did release new, more accurate calculations. But it wasn’t the new calculations, by themselves, that made a difference. Instead, it was a shift in what kind of calculations they used…or even more specifically, what kind of calculations they trusted.

Parts of the calculation of g-2 can be done with Feynman diagrams, those photogenic squiggles you see on physicists’ blackboards. That part is very precise, and not especially controversial. However, Feynman diagrams only work well when forces between particles are comparatively weak. They’re great for electromagnetism, even better for the weak nuclear force. But for the strong nuclear force, the one that holds protons and neutrons together, you often need a different method.

For g-2, that used to be done via a “data-driven” method. Physicists measured different things, particles affected by the strong nuclear force in different ways, and used that to infer how the strong force would affect g-2. By getting a consistent picture from different experiments, they were reasonably confident that they had the right numbers.

Back in 2020, though, a challenger came to the scene, with another method. Called lattice QCD, this method involves building gigantic computer simulations of the effect of the strong force. People have been doing lattice QCD since the 1970’s, and the simulations have been getting better and better, until in 2020, a group managed to calculate the piece of the g-2 calculation that had until then been done by the data-driven method.

The lattice group found a very different result than what had been found previously. Instead of a wild disagreement with experiment, their calculation agreed. According to them, everything was fine, the muon g-2 was behaving exactly as the Standard Model predicted.

For some of us, that’s where the mystery ended. Clearly, something must be wrong with the data-driven method, not with the Standard Model. No more muon puzzle.

But the data-driven method wasn’t just a guess, it was being used for a reason. A significant group of physicists found the arguments behind it convincing. Now, there was a new puzzle: figuring out why the data-driven method and lattice QCD disagree.

Five years later, has that mystery been solved? Is that, finally, what the headlines are about?

Again, not really, no.

The theorists associated with the experiment have decided to trust lattice QCD, not the data-driven method. But they don’t know what went wrong, exactly.

Instead, they’ve highlighted cracks in the data-driven method. The way the data-driven method works, it brings together different experiments to try to get a shared picture. But that shared picture has started to fall apart. A new measurement by a different experiment doesn’t fit into the system: the data-driven method now “has tensions”, as physicists say. It’s no longer possible to combine all experiments into a shared picture they way they used to. Meanwhile, lattice QCD has gotten even better, reaching even higher precision. From the perspective of the theorists associated with the muon g-2 experiment, switching methods is now clearly the right call.

But does that mean they solved the puzzle?

If you were confident that lattice QCD is the right approach, then the puzzle was already solved in 2020. All that changed was the official collaboration finally acknowledging that.

And if you were confident that the data-driven method was the right approach, then the puzzle is even worse. Now, there are tensions within the method itself…but still no explanation of what went wrong! If you had good reasons to think the method should work, you still have those good reasons. Now you’re just…more puzzled.

I am reminded of another mystery, a few years back, when an old experiment announced a dramatically different measurement for the mass of the W boson. Then, I argued the big mystery was not how the W boson’s mass had changed (it hadn’t), but how they came to be so confident in a result so different from what others, also confidently, had found. In physics, our confidence is encoded in numbers, estimated and measured and tested and computed. If we’re not estimating that confidence correctly…then that’s the real mystery, the real puzzle. One much more important to solve.

Also, I had two more pieces out this week! In Quanta I have a short explainer about bosons and fermions, while at Ars Technica I have a piece about machine learning at the LHC. I may have a “bonus info” post on the latter at some point, I have to think about whether I have enough material for it.

I’m working on a much longer post on phases and phase transitions for next week1, but in the meantime, let me share with you some cool facts about water and its “phases.”

\We all know about solids, liquids, and gases from school. Heat up ice, and you get water; heat up water, and you get vapor. We may even have been slightly baffled if we saw this phase diagram with “pressure” added to the mix

Thanks for reading Quantum Matters! Subscribe for free to receive new posts and support my work.

I see here a solid phase, a liquid phase, and a gas phase, but what is this “Critical point”? If you tune your temperature and pressure just right you can smoothly cross over from liquid to gas without ever undergoing a phase transition. Without getting into the molecular details, we can think of phases as particular valleys between mountains, and water wants to reach the absolute lowest point. Sometimes there are two valleys, but one is lower, and sometimes there is just one valley.

In fact, this “number of valleys” is why we see this odd behavior. If we sit at 100 degrees C and decrease or increase the pressure, there are two energy minima2—two valleys. At small pressure, the deepest valley is on the gas side, and at large pressure, the deepest valley is on the liquid side. As you then tune pressure across that one-bar point, one valley gets deeper than the other—it’s the true minimum! Yet, to get from one valley to the next, you need some energy to get you over that mountain in between. That’s the phase transition. However, that's not the only option. As the temperature increases, the mountain in between gets smaller and smaller until, at the critical point, it finally disappears, and the two valleys merge.

Without two distinguished valleys, there is no need to scale the mountain and no need for a phase transition. Liquid smoothly and easily becomes gas. At the temperatures above the critical point, you cannot meaningfully distinguish water and gas. OK, so perhaps we only have two phases?

Not quite; look at this more fleshed-out version of the phase diagram:

When ice forms, it adopts a low-energy crystal structure. However, there are numerous crystal structures to choose from. In fact, as you change pressure and temperature, it can completely reorganize how the ice bonds together into a crystal. This leads to over 20 phases of ice, labeled by some of the Roman numerals above.3

Then what are the phases? Solids undergo their own phase transitions—structural phase transitions. Are these not phases of matter? If they are, then we have already exceeded our three phases of matter just within water. But phases go beyond temperature and pressure. They also possess a multitude of interesting properties, particularly at that critical point. We'll cover some of that in detail next week.

1

We’ll be making our own phase! Related, of course, to a known phase transition.

2

For most of the phase diagram, there is one absolute minimum, and the other is a “metastable” or local minimum.

3

For those interested, this Wikipedia article has a lot of information on the phases of ice.

© Justin H Wilson at June 27, 2025 02:31 PM