Problem 1026 on the Erdős problem web site recently got solved through an interesting combination of existing literature, online collaboration, and AI tools. The purpose of this blog post is to try to tell the story of this collaboration, and also to supply a complete proof.

The original problem of Erdős, posed in 1975, is rather ambiguous. Erdős starts by recalling his famous theorem with Szekeres that says that given a sequence of distinct real numbers, one can find a subsequence of length which is either increasing or decreasing; and that one cannot improve the to , by considering for instance a sequence of blocks of length , with the numbers in each block decreasing, but the blocks themselves increasing. He also noted a result of Hanani that every sequence of length can be decomposed into the union of monotone sequences. He then wrote “As far as I know the following question is not yet settled. Let be a sequence of distinct numbers, determine

This problem was added to the Erdős problem site on September 12, 2025, with a note that the problem was rather ambiguous. For any fixed , this is an explicit piecewise linear function of the variables that could be computed by a simple brute force algorithm, but Erdős was presumably seeking optimal bounds for this quantity under some natural constraint on the . The day the problem was posted, Desmond Weisenberg proposed studying the quantity , defined as the largest constant such that

Though not stated on the web site, one can formulate this problem in game theoretic terms. Suppose that Alice has a stack of coins for some large . She divides the coins into piles of consisting of coins each, so that . She then passes the piles to Bob, who is allowed to select a monotone subsequence of the piles (in the weak sense) and keep all the coins in those piles. What is the largest fraction of the coins that Bob can guarantee to keep, regardless of how Alice divides up the coins? (One can work with either a discrete version of this problem where the are integers, or a continuous one where the coins can be split fractionally, but in the limit the problems can easily be seen to be equivalent.)

AI-generated images continue to be problematic for a number of reasons, but here is one such image that somewhat manages at least to convey the idea of the game:

For small , one can work out by hand. For , clearly : Alice has to put all the coins into one pile, which Bob simply takes. Similarly : regardless of how Alice divides the coins into two piles, the piles will either be increasing or decreasing, so in either case Bob can take both. The first interesting case is . Bob can again always take the two largest piles, guaranteeing himself of the coins. On the other hand, if Alice almost divides the coins evenly, for instance into piles for some small , then Bob cannot take all three piles as they are non-monotone, and so can only take two of them, allowing Alice to limit the payout fraction to be arbitrarily close to . So we conclude that .

An hour after Desmond’s comment, Stijn Cambie noted (though not in the language I used above) that a similar construction to the one above, in which Alice divides the coins into pairs that are almost even, in such a way that the longest monotone sequence is of length , gives the upper bound . It is also easy to see that is a non-increasing function of , so this gives a general bound . Less than an hour after that, Wouter van Doorn noted that the Hanani result mentioned above gives the lower bound , and posed the problem of determining the asymptotic limit of as , given that this was now known to range between and . This version was accepted by Thomas Bloom, the moderator of the Erdős problem site, as a valid interpretation of the original problem.

The next day, Stijn computed the first few values of exactly:

The problem then lay dormant for almost two months, until December 7, 2025, in which Boris Alexeev, as part of a systematic sweep of the Erdős problems using the AI tool Aristotle, was able to get this tool to autonomously solve this conjecture in the proof assistant language Lean. The proof converted the problem to a rectangle-packing problem.

This was one further addition to a recent sequence of examples where an Erdős problem had been automatically solved in one fashion or another by an AI tool. Like the previous cases, the proof turned out to not be particularly novel. Within an hour, Koishi Chan gave an alternate proof deriving the required bound from the original Erdős-Szekeres theorem by a standard “blow-up” argument which we can give here in the Alice-Bob formulation. Take a large , and replace each pile of coins with new piles, each of size , chosen so that the longest monotone subsequence in this collection is . Among all the new piles, the longest monotone subsequence has length . Applying Erdős-Szekeres, one concludes the bound

Once this proof was found, it was natural to try to see if it had already appeared in the literature. AI deep research tools have successfully located such prior literature in the past, but in this case they did not succeed, and a more “old-fashioned” Google Scholar job turned up some relevant references: a 2016 paper by Tidor, Wang and Yang contained this precise result, citing an earlier paper of Wagner as inspiration for applying “blowup” to the Erdős-Szekeres theorem.

But the story does not end there! Upon reading the above story the next day, I realized that the problem of estimating was a suitable task for AlphaEvolve, which I have used recently as mentioned in this previous post. Specifically, one could task to obtain upper bounds on by directing it to produce real numbers (or integers) summing up to a fixed sum (I chose ) with a small a value of as possible. After an hour of run time, AlphaEvolve produced the following upper bounds on for , with some intriguingly structured potential extremizing solutions:

Proposition 1 If and , then .

Proof: Consider a sequence of numbers clustered around the “red number” and “blue number” , consisting of blocks of “blue” numbers, followed by blocks of “red” numbers, and then further blocks of “blue” numbers. When , one should take all blocks to be slightly decreasing within each block, but the blue blocks should be are increasing between each other, and the red blocks should also be increasing between each other. When , all of these orderings should be reversed. The total number of elements is indeed

Here is a figure illustrating the above construction in the case (obtained after starting with a ChatGPT-provided file and then manually fixing a number of placement issues):

Here is a plot of (produced by ChatGPT Pro), showing that it is basically a piecewise linear approximation to the square root function:

Shortly afterwards, Lawrence Wu clarified the connection between this problem and a square packing problem, which was also due to Erdős (Problem 106). Let be the least number such that, whenever one packs squares of sidelength into a square of sidelength , with all sides parallel to the coordinate axes, one has

Proposition 2 For any , one has

Proof: Given and , let be the maximal sum over all increasing subsequences ending in , and be the maximal sum over all decreasing subsequences ending in . For , we have either (if ) or (if ). In particular, the squares and are disjoint. These squares pack into the square , so by definition of , we have

This idea of using packing to prove Erdős-Szekeres type results goes back to a 1959 paper of Seidenberg, although it was a discrete rectangle-packing argument that was not phrased in such an elegantly geometric form. It is possible that Aristotle was “aware” of the Seidenberg argument via its training data, as it had incorporated a version of this argument in its proof.

Here is an illustration of the above argument using the AlphaEvolve-provided example

for to convert it to a square packing (image produced by ChatGPT Pro):

At this point, Lawrence performed another AI deep research search, this time successfully locating a paper from just last year by Baek, Koizumi, and Ueoro, where they show that

Theorem 3 For any , one has

which, when combined with a previous argument of Praton, implies

Theorem 4 For any and with , one has

There just remained the issue of putting everything together. I did feed all of the above information into a large language model, which was able to produce a coherent proof of (1) assuming the results of Baek-Koizumi-Ueoro and Praton. Of course, LLM outputs are prone to hallucination, so it would be preferable to formalize that argument in Lean, but this looks quite doable with current tools, and I expect this to be accomplished shortly. But I was also able to reproduce the arguments of Baek-Koizumi-Ueoro and Praton, which I include below for completeness.

Proof: (Proof of Theorem 3, adapted from Baek-Koizumi-Ueoro) We can normalize . It then suffices to show that if we pack the length torus by axis-parallel squares of sidelength , then

Pick . Then we have a grid

UPDATE: Actually, the above argument also proves Theorem 4 with only minor modifications. Nevertheless, we give the original derivation of Theorem 4 using the embedding argument of Praton below for sake of completeness.

Proof: (Proof of Theorem 4, adapted from Praton) We write with . We can rescale so that the square one is packing into is . Thus, we pack squares of sidelength into , and our task is to show that

One striking feature of this story for me is how important it was to have a diverse set of people, literature, and tools to attack this problem. To be able to state and prove the precise formula for required multiple observations, including some version of the following:

• The sequence can be numerically computed as a sequence of rational numbers.
• When appropriately normalized and arranged, visible patterns in this sequence appear that allow one to conjecture the form of the sequence.
• This problem is a weighted version of the Erdős-Szekeres theorem.
• Among the many proofs of the Erdős-Szekeres theorem is the proof of Seidenberg in 1959, which can be interpreted as a discrete rectangle packing argument.
• This problem can be reinterpreted as a continuous square packing problem, and in fact is closely related to (a generalized axis-parallel form of) Erdős problem 106, which concerns such packings.
• The axis-parallel form of Erdős problem 106 was recently solved by Baek-Koizumi-Ueoro.
• The paper of Praton shows that Erdős Problem 106 implies the generalized version needed for this problem. This implication specializes to the axis-parallel case.

Another key ingredient was the balanced AI policy on the Erdős problem website, which encourages disclosed AI usage while strongly discouraging undisclosed use. To quote from that policy: “Comments prepared with the assistance of AI are permitted, provided (a) this is disclosed, (b) the contents (including mathematics, code, numerical data, and the existence of relevant sources) have been carefully checked and verified by the user themselves without the assistance of AI, and (c) the comment is not unreasonably long.”

This piece of marketing doggerel from McDonald’s, concerning their Xmas-special Grinch meal, is the most foully defiling attempt to imitate Dr. Seuss’s verse I’ve ever encountered.

How I know zombies aren’t real is that, if zombies were real, we would already have read about the McDonald’s PR employee who wrote this abomination being brutally eviscerated by a vengeful Dead Seuss.

In school, kids learn about different types of energy. They learn about solar energy and wind energy, nuclear energy and chemical energy, electrical energy and mechanical energy, and potential energy and kinetic energy. They learn that energy is conserved, that it can never be created or destroyed, but only change form. They learn that energy makes things happen, that you can use energy to do work, that energy is different from matter.

Some, between good teaching and good students, manage to impose order on the jumble of concepts and terms. Others end up envisioning the whole story a bit like Pokemon, with different types of some shared “stuff”.

Energy isn’t “stuff”, though. So what is it? What relates all these different types of things?

Energy is something which is conserved.

The mathematician Emmy Noether showed that, when the laws of physics are symmetrical, they come with a conserved quantity. For example, because the laws of the physics are the same from place to place, momentum is conserved. Similarly, because the laws of physics are the same from one time to another, Noether’s theorem states that there must be some quantity related to time, some number we can calculate, that is conserved, even as other things change. We call that number energy.

If energy is that simple, why are there all those types?

Energy is a number we can calculate. It’s a number we can calculate for different things. If you have a detailed description of how something in physics works, you can use that description to calculate that thing’s energy. In school, you memorize formulas like and . These are all formulas that, with a bit more knowledge, you could calculate. They are the things that, for a something that meets the conditions, are conserved. They are things that, according to Noether’s theorem, stay the same.

Because of this, you shouldn’t think of energy as a substance, or a fuel. Energy is something we can do: we physicists, and we students of physics. We can take a physical system, and see what about it ought to be conserved. Energy is an action, a calculation, a conceptual tool that can be used to make predictions.

Most things are, in the end.

read more

The Baltimore Orioles have signed Madison Mallards legend Pete Alonso to a 5-year contract. $155 million. The biggest deal since (ulp) Chris Davis. He’ll wear Rafael Palmeiro and Anthony Santander’s #25. And Jay Gibbons’.

The fanbase is over the moon that we finally signed somebody. I get it, after so many years with no big contracts. But I also don’t get it. One of the Orioles podcasters I listen to said “This guy gets you 40, maybe 50 home runs a year.” Does he? Last year he hit 38, the year before that 34. He’s hit 50 once and only once, his rookie year in 2019. He does hit home runs! More than anybody else on the Orioles. But beyond that? He sorta kinda gets on base and he’s a terrible defensive first baseman. He’s not likely to be the Orioles’ best player. He would have been third best last year, after Gunnar Henderson and Trevor Rogers. A team whose best player is Pete Alonso is not a contending team.

And yet, for all this, I’m cheered by this deal. There is no Mookie Betts or Juan Soto or Shohei Ohtani to sign right now. And the Orioles with Alonso are incrementally better than the Orioles without Alonso. What you have to do to be good, in 2025, is be willing to spend a lot of money to incrementally improve your team, with guys, who, if not superstars, are at least plausible regular stars. The Orioles have not been willing to do that for a long time, and now, maybe, they are.

Also, of course, the team now has five guys, Rutschmann, Basallo, Mountcastle, Mayo, and Alonso who are limited to three positions, C,1B,DH. Alonso being here means one or more of Mountcastle and Mayo probably get traded for pitching. So if you spend the money to get Alonso playing every day at 1B (and one thing I’ll say for the guy, he does play every day) the improvement over Mayo playing there every day maybe isn’t that great; but if instead of Mayo sitting on the bench you turn him into a #2 or #3 starter, now you’re starting to get some real value for your $31m a year. (You know, now that we have more righthanded punch in the lineup, does Taylor Ward become a trade chip for pitching? Maybe we could talk to the Angels, they have a young guy Grayson Rodriguez with a spotty injury history but a lot of upside…)

But look, even if we do add pitching: you take the players on the 2025 Orioles, you add a poor defender who wears #25 and hits 40 HR and an ace starting pitcher, and you know what you have? The 2024 Orioles. Who were not that great! We did the tank, we did the rebuild, we have most of an entire lineup of players who were high draft picks and blue-chip prospects just a few years ago. If those guys play like they were supposed to play, this is a really good team. If they play like they played in 2025, $155m is not gonna help.

Not long ago William MacAskill, the founder of the Effective Altruist movement, visited Austin, where I got to talk with him in person for the first time. I was a fan of his book What We Owe the Future, and found him as thoughtful and eloquent face-to-face as I did on the page. Talking to Will inspired me to write the following short reflection on how I should spend my time, which I’m now sharing in case it’s of interest to anyone else.

By inclination and temperament, I simply seek the clearest possible understanding of reality.  This has led me to spend time on (for example) the Busy Beaver function and the P versus NP problem and quantum computation and the foundations of quantum mechanics and the black hole information puzzle, and on explaining whatever I’ve understood to others.  It’s why I became a professor.

But the understanding I’ve gained also tells me that I should try to do things that will have huge positive impact, in what looks like a pivotal and even terrifying time for civilization.  It tells me that seeking understanding of the universe, like I’ve been doing, is probably nowhere close to optimizing any values that I could defend.  It’s self-indulgent, a few steps above spending my life learning to solve Rubik’s Cube as quickly as possible, but only a few.  Basically, it’s the most fun way I could make a good living and have a prestigious career, so it’s what I ended up doing.  I should be skeptical that such a course would coincidentally also maximize the good I can do for humanity.

Instead I should plausibly be figuring out how to make billions of dollars, in cryptocurrency or startups or whatever, and then spending it in a way that saves human civilization, for example by making AGI go well.  Or I should be convincing whatever billionaires I know to do the same.  Or executing some other galaxy-brained plan.  Even if I were purely selfish, as I hope I’m not, still there are things other than theoretical computer science research that would bring more hedonistic pleasure.  I’ve basically just followed a path of least resistance.

On the other hand, I don’t know how to make billions of dollars.  I don’t know how to make AGI go well.  I don’t know how to influence Elon Musk or Sam Altman or Peter Thiel or Sergey Brin or Mark Zuckerberg or Marc Andreessen to do good things rather than bad things, even when I have gotten to talk to some of them.  Past attempts in this direction by extremely smart and motivated people—for example, those of Eliezer Yudkowsky and Sam Bankman-Fried—have had, err, uneven results, to put it mildly.  I don’t know why I would succeed where they failed.

Of course, if I had a better understanding of reality, I might know how better to achieve prosocial goals for humanity.  Or I might learn why they were actually the wrong goals, and replace them with better goals.  But then I’m back to the original goal of understanding reality as clearly as possible, with the corresponding danger that I spend my time learning to solve Rubik’s Cube faster.

AI promises to revolutionize the way we do science, which raises a central technological question of our time: Can classical AI understand all natural phenomena, or are some fundamentally beyond its reach? Many proponents of artificial intelligence argue that any pattern that can be generated or found in nature can be efficiently discovered and modeled by a classical learning algorithm, implying that AI is a universal and sufficient tool for science.

The word “classical” is important here to contrast with quantum computation. Nature is quantum mechanical, and the insights of Shor’s algorithm [1] along with quantum error correction [2,3,4] teach us that there are quantum systems, at least ones that have been heavily engineered, that can have trajectories that are fundamentally unpredictable¹ by any classical algorithm, including AI. This opens the possibility that there are complex quantum phenomena occurring naturally in our universe where classical AI is insufficient, and we need a quantum computer in order to model them.

This essay uses the perspective of computational complexity to unpack this nuanced question. We begin with quantum sampling, arguing that despite clear quantum supremacy, it does not represent a real hurdle for AI to predict quantum phenomena. We then shift to the major unsolved problems in quantum physics and quantum chemistry, examining how quantum computers could empower AI in these domains. Finally, we’ll consider the possibility of truly complex quantum signals in nature, where quantum computers might prove essential for prediction itself.

Quantum Sampling

In 2019 Google demonstrated quantum supremacy on a digital quantum device [5], and in 2024 their latest chip performed a task in minutes where our best classical computers would take 10^25 years [6]. The task they performed is to prepare a highly entangled many-body quantum state and to sample from the corresponding distribution over classical configurations. Quantum supremacy on such sampling problems is on firm ground, with results in complexity theory backing up the experimental claims [7].² Moreover, the classical hardness of quantum sampling appears to be generic in quantum physics. A wide range of quantum systems will generate highly entangled many-body states where sampling becomes classically hard.

However, quantum sampling alone does not refute the universality of classical AI. The output of quantum sampling often appears completely featureless, which cannot be verified by any classical or quantum algorithm, or by any process in our universe for that matter. For example, running the exact same sampling task a second time will produce a list of configurations that will appear unrelated to the original. In order for a phenomenon to be subject to scientific prediction, there must be an experiment that can confirm or deny the prediction. So if quantum sampling has no features that can be experimentally verified, there is nothing to predict, and no pattern for the AI to discover and model.

Quantum Chemistry and Condensed Matter Physics

There are many unsolved problems in quantum chemistry and condensed matter physics that are inaccessible using our best classical simulation algorithms and supercomputers. For example, these occur in the strongly correlated regime of electronic structure in quantum chemistry, and around low-temperature phase transitions of condensed matter systems. We do not understand the electronic structure of FeMoco, the molecule responsible for nitrogen fixation in the nitrogenase enzyme, nor do we understand the phase diagram of the 2D Fermi-Hubbard lattice and whether or not it exhibits superconductivity.

It is possible there are no fundamental barriers for a sufficiently advanced AI to solve these problems. Researchers in the field have achieved major breakthroughs using neural networks to predict complex biological structures like protein folding. One could imagine similar specialized AI models that predict the electronic structure of molecules, or that predict quantum phases of matter. Perhaps the main reason it is currently out of reach is a lack of sufficient training data. Here lies a compelling opportunity for quantum computing: The only feasible way to generate an abundance of accurate training data may be to use a quantum computer, since physical experiments are too difficult, too unreliable, and too slow.

How should we view these problems in physics and chemistry from the perspective of computational complexity? Physicists and chemists often consider systems with a fixed number of parameters, or even single instances. Although computing physical quantities may be extremely challenging, single instances cannot form computationally hard problems, since ultimately only a constant amount of resources is required to solve it. Systems with a fixed number of parameters often behave similarly, since physical quantities tend to depend smoothly on the parameters which allows for extrapolation and learning [8]. Here we can recognize a familiar motif from machine learning: Ab initio prediction is challenging, but prediction becomes efficient after learning from data. Quantum computers are useful for generating training data, but then AI is able to learn from this data and perform efficient inference.

Truly Complex Quantum Signals

While AI might be able to learn much of the patterns of physics and chemistry from quantum-generated data, there remains a deeper possibility: The quantum universe may produce patterns that AI cannot compress and understand. If there are quantum systems that display signals that are truly classically complex, then predicting the pattern will require a quantum computer at inference time, not only in training.

We’ll now envision how such a signal could arise. Imagine a family of quantum systems of arbitrary size N, and at each size N there is a number of independent parameters that is polynomial in N, for example the coefficients of a Hamiltonian or the rotation angles of a quantum circuit. Suppose the system has some physical feature whose signal we would like to compute as a function of the parameters, and this signal has the following properties:

• (Signal) There is a quantum algorithm that efficiently computes the signal. For example, the signal cannot be exponentially small in N.
• (Verifiable) The signal is verifiable, at least by an ideal quantum computer. For example, the task could be to compute an expectation value.
• (Typically complex) When the parameters are chosen randomly, the signal is computationally hard to classically compute in the average case.

If these properties hold, then it’s possible that no machine learning model using a polynomial amount of classical compute can perform the task, even with the help of training data.

The requirement of verifiability by a quantum process ensures that the signal being computed is a robust phenomenon where there is some “fact of the matter”, and a prediction can be confirmed or denied by nature. For example, this holds for any task where the output is the expectation value of some observable. The average-case hardness ensures that hard instances really exist and can be easily generated, rather than only existing in some abstract worst-case that cannot be instantiated.

There is a connection between the verifiability of a computation and its utility to us. Suppose we use a computer to help us design a high-temperature superconductor. If our designed material indeed works as a high-temperature superconductor when fabricated, this forms a verification of the predictions made by our computer. Utility implies verifiability, and likewise, unverifiable computations cannot be useful. However, since nature is quantum, a computation need not be classically verifiable in order to be useful, but only quantumly verifiable. In our high-temperature superconductor example, nature has verified our computer by performing a quantum process.

Making progress

John Preskill’s “entanglement frontier” seeks to understand the collective behavior of many interacting quantum particles [10]. In order to shed light on the fundamental limits of classical AI and the utility of quantum computers in this regime, we must understand if the exponential Hilbert space of quantum theory remains mostly hidden, or if it reveals itself in observable phenomena. The search for classically complex signals forms an exciting research program for making progress. Google recently performed the first demonstration of a classically complex signal on a quantum device: The out-of-time-order correlators³ of random quantum circuits [9]. We can seek to find more such examples, first in abstract models, and then in the real world, to understand how abundant they are in nature.

1. Under widely accepted cryptographic assumptions. ︎
2. Classical computers cannot perform quantum sampling unless the polynomial hierarchy collapses. ︎
3. More precisely, the higher moments of the out-of-time-order correlator. ︎

References

[1] Shor, Peter W. “Algorithms for quantum computation: discrete logarithms and factoring.” Proceedings 35th annual symposium on foundations of computer science. Ieee, 1994.

[2] Shor, Peter W. “Scheme for reducing decoherence in quantum computer memory.” Physical review A 52.4 (1995): R2493.

[3] Shor, Peter W. “Fault-tolerant quantum computation.” Proceedings of 37th conference on foundations of computer science. IEEE, 1996.

[4] Kitaev, A. Yu. “Fault-tolerant quantum computation by anyons.” Annals of physics 303.1 (2003): 2-30.

[5] Arute, Frank, et al. “Quantum supremacy using a programmable superconducting processor.” Nature 574.7779 (2019): 505-510.

[6] Morvan, Alexis, et al. “Phase transitions in random circuit sampling.” Nature 634.8033 (2024): 328-333.

[7] Aaronson, Scott, and Alex Arkhipov. “The computational complexity of linear optics.” Proceedings of the forty-third annual ACM symposium on Theory of computing. 2011.

[8] Huang, Hsin-Yuan, et al. “Provably efficient machine learning for quantum many-body problems.” Science 377.6613 (2022): eabk3333.

[9] Abanin, Dmitry A., et al. “Constructive interference at the edge of quantum ergodic dynamics.” arXiv preprint arXiv:2506.10191 (2025).

[10] Preskill, John. “Quantum computing and the entanglement frontier.” arXiv preprint arXiv:1203.5813 (2012).

Matthew Bolan, Joachim Breitner, Jose Brox, Nicholas Carlini, Mario Carneiro, Floris van Doorn, Martin Dvorak, Andrés Goens, Aaron Hill, Harald Husum, Hernán Ibarra Mejia, Zoltan Kocsis, Bruno Le Floch, Amir Livne Bar-on, Lorenzo Luccioli, Douglas McNeil, Alex Meiburg, Pietro Monticone, Pace P. Nielsen, Emmanuel Osalotioman Osazuwa, Giovanni Paolini, Marco Petracci, Bernhard Reinke, David Renshaw, Marcus Rossel, Cody Roux, Jérémy Scanvic, Shreyas Srinivas, Anand Rao Tadipatri, Vlad Tsyrklevich, Fernando Vaquerizo-Villar, Daniel Weber, Fan Zheng, and I have just uploaded to the arXiv our preprint The Equational Theories Project: Advancing Collaborative Mathematical Research at Scale. This is the final report for the Equational Theories Project, which was proposed in this blog post and also showcased in this subsequent blog post. The aim of this project was to see whether one could collaboratively achieve a large-scale systematic exploration of a mathematical space, which in this case was the implication graph between 4694 equational laws of magmas. A magma is a set equipped with a binary operation (or, equivalently, a multiplication table). An equational law is an equation involving this operation and a number of indeterminate variables. Some examples of equational laws, together with the number that we assigned to that law, include

• (Trivial law):
• (Singleton law):
• (Commutative law):
• (Central groupoid law):
• (Associative law):

The aim of the project was to work out which of these laws imply which others. For instance, all laws imply the trivial law , and conversely the singleton law implies all the others. On the other hand, the commutative law does not imply the associative law (because there exist magmas that are commutative but not associative), nor is the converse true. All in all, there are implications of this type to settle; most of these are relatively easy and could be resolved in a matter of minutes by an expert in college-level algebra, but prior to this project, it was impractical to actually do so in a manner that could be feasibly verified. Also, this problem is known to become undecidable for sufficiently long equational laws. Nevertheless, we were able to resolve all the implications informally after two months, and have them completely formalized in Lean after a further five months.

After a rather hectic setup process (documented in this personal log), progress came in various waves. Initially, huge swathes of implications could be resolved first by very simple-minded techniques, such as brute-force searching all small finite magmas to refute implications; then, automated theorem provers such as Vampire or Mace4 / Prover9 were deployed to handle a large fraction of the remainder. A few equations had existing literature that allowed for many implications involving them to be determined. This left a core of just under a thousand implications that did not fall to any of the “cheap” methods, and which occupied the bulk of the efforts of the project. As it turns out, all of the remaining implications were negative; the difficulty was to construct explicit magmas that obeyed one law but not another. To do this, we discovered a number of general constructions of magmas that were effective at this task. For instance:

• Linear models, in which the carrier was a (commutative or noncommutative) ring and the magma operation took the form for some coefficients , turned out to resolve many cases.
• We discovered a new invariant of an equational law, which we call the “twisting semigroup” of that law, which also allowed us to construct further examples of magmas that obeyed one law but not another , by starting with a base magma that obeyed both laws, taking a Cartesian power of that magma, and then “twisting” the magma operation by certain permutations of designed to preserve but not .
• We developed a theory of “abelian magma extensions”, similar to the notion of an abelian extension of a group, which allowed us to flexibly build new magmas out of old ones in a manner controlled by a certain “magma cohomology group ” which were tractable to compute, and again gave ways to construct magmas that obeyed one law but not another .
• Greedy methods, in which one fills out an infinite multiplication table in a greedy manner (somewhat akin to naively solving a Sudoku puzzle), subject to some rules designed to avoid collisions and maintain a law , as well as some seed entries designed to enforce a counterexample to a separate law . Despite the apparent complexity of this method, it can be automated in a manner that allowed for many outstanding implications to be refuted.
• Smarter ways to utilize automated theorem provers, such as strategically adding in additional axioms to the magma to help narrow the search space, were developed over the course of the project.

Even after applying all these general techniques, though, there were about a dozen particularly difficult implications that resisted even these more powerful methods. Several ad hoc constructions were needed in order to understand the behavior of magmas obeying such equations as E854, E906, E1323, E1516, and E1729, with the latter taking months of effort to finally solve and then formalize.

A variety of GUI interfaces were also developed to facilitate the collaboration (most notably the Equation Explorer tool mentioned above), and several side projects were also created within the project, such as the exploration of the implication graph when the magma was also restricted to be finite. In this case, we resolved all of the implications except for one (and its dual):

Problem 1 Does the law (E677) imply the law (E255) for finite magmas?

See this blueprint page for some partial results on this problem, which we were unable to resolve even after months of effort.

Interestingly, modern AI tools did not play a major role in this project (but it was largely completed in 2024, before the most recent advanced models became available); while they could resolve many implications, the older “good old-fashioned AI” of automated theorem provers were far cheaper to run and already handled the overwhelming majority of the implications that the advanced AI tools could. But I could imagine that such tools would play a more prominent role in future similar projects.

Just a couple more notes on this setup so I don’t forget.

First of all, I set this up as a problem about G acting on G/H, i.e. G acting transitively on a set. But the transitivity is actually not critical. The group could be trivial! If we call the set Σ, then G could be the trivial group, and we’d just be asking for large subsets S of Σ such that S^m was contained in some specified subset R of Σ^m. There are perfectly questions like that! I’ve seen this come up for instance when Σ is k and R is an algebraic subvariety of k^m. So maybe you could think of the setup of the blogpost as “questions like that where the problem admits a really big automorphism group, maybe so big that the elements of Σ are homogeneous for it.” I do think the transitivity cuts you down to a particularly congenial and natural class of such problems; “problems with no moduli” if you like.

Yet one more note. I think one could work with more general tensor categories than the category of sets. For instance, one could take V to be a representation of G, and let R be a subrepresentation of V^{⊗m}, and ask: how large can a subspace (not a subrepresentation) W of V be with the property that W^{⊗m} lies in R? And tensor powers are actually not necessarily the most natural thing to do to V; probably rather than choosing m and tensoring m times, we should choose a partition λ and apply the λ Schur functor. I assumed that once again lots of standard problems would fall out of this setup, but to be honest, I couldn’t think of any. Can you?

I guess here’s one baby case. Let V be the standard permutation representation of S_n. Then Sym^d V is homogeneous forms in n variables with S_n permuting coordinates. Let R be the subrepresentation of forms F such that the coefficients of X_i^d sum to 0. Then a subspace W of linear forms such that Sym^d W lies in R is just (I think) a linear subspace of the diagonal hypersurface sum x_i^d = 0 in P^{n-1}, and such a subspace has dimension at most n/2 or so I think (pair up coordinates to make x_i^d = – x_{i+1}^d; maybe you have to be a little more clever if d is even and -1 doesn’t have a square root?)

The following is based on a talk that I gave (remotely) at the UK AI Safety Institute Alignment Workshop on October 29, and which I then procrastinated for more than a month in writing up. Enjoy!

Thanks for having me! I’m a theoretical computer scientist. I’ve spent most of my career for ~25 years studying the capabilities and limits of quantum computers. But for the past 3 or 4 years, I’ve also been moonlighting in AI alignment. This started with a 2-year leave at OpenAI, in what used to be their Superalignment team, and it’s continued with a 3-year grant from Coefficient Giving (formerly Open Philanthropy) to build a group here at UT Austin, looking for ways to apply theoretical computer science to AI alignment. Before I go any further, let me mention some action items:

• Our Theory and Alignment group is looking to recruit new PhD students this fall! You can apply for a PhD at UTCS here; the deadline is quite soon (December 15). If you specify that you want to work with me on theory and AI alignment (or on quantum computing, for that matter), I’ll be sure to see your application. For this, there’s no need to email me directly.
  
• We’re also looking to recruit one or more postdoctoral fellows, working on anything at the intersection of theoretical computer science and AI alignment! Fellowships to start in Fall 2026 and continue for two years. If you’re interested in this opportunity, please email me by January 15 to let me know you’re interested. Include in your email a CV, 2-3 of your papers, and a research statement and/or a few paragraphs about what you’d like to work on here. Also arrange for two recommendation letters to be emailed to me. Please do this even if you’ve contacted me in the past about a potential postdoc.
  
• While we seek talented people, we also seek problems for those people to solve: any and all CS theory problems motivated by AI alignment! Indeed, we’d like to be a sort of theory consulting shop for the AI alignment community. So if you have such a problem, please email me! I might even invite you to speak to our group about your problem, either by Zoom or in person.

Our search for good problems brings me nicely to the central difficulty I’ve faced in trying to do AI alignment research. Namely, while there’s been some amazing progress over the past few years in this field, I’d describe the progress as having been almost entirely empirical—building on the breathtaking recent empirical progress in AI capabilities. We now know a lot about how to do RLHF, how to jailbreak and elicit scheming behavior, how to look inside models and see what’s going on (interpretability), and so forth—but it’s almost all been a matter of trying stuff out and seeing what works, and then writing papers with a lot of bar charts in them.

The fear is of course that ideas that only work empirically will stop working when it counts—like, when we’re up against a superintelligence. In any case, I’m a theoretical computer scientist, as are my students, so of course we’d like to know: what can we do?

After a few years, alas, I still don’t feel like I have any systematic answer to that question. What I have instead is a collection of vignettes: problems I’ve come across where I feel like a CS theory perspective has helped, or plausibly could help. So that’s what I’d like to share today.

Probably the best-known thing I’ve done in AI safety is a theoretical foundation for how to watermark the outputs of Large Language Models. I did that shortly after starting my leave at OpenAI—even before ChatGPT came out. Specifically, I proposed something called the Gumbel Softmax Scheme, by which you can take any LLM that’s operating at a nonzero temperature—any LLM that could produce exponentially many different outputs in response to the same prompt—and replace some of the entropy with the output of a pseudorandom function, in a way that encodes a statistical signal, which someone who knows the key of the PRF could later detect and say, “yes, this document came from ChatGPT with >99.9% confidence.” The crucial point is that the quality of the LLM’s output isn’t degraded at all, because we aren’t changing the model’s probabilities for tokens, but only how we use the probabilities. That’s the main thing that was counterintuitive to people when I explained it to them.

Unfortunately, OpenAI never deployed my method—they were worried (among other things) about risk to the product, customers hating the idea of watermarking and leaving for a competing LLM. Google DeepMind has deployed something in Gemini extremely similar to what I proposed, as part of what they call SynthID. But you have to apply to them if you want to use their detection tool, and they’ve been stingy with granting access to it. So it’s of limited use to my many faculty colleagues who’ve been begging me for a way to tell whether their students are using AI to cheat on their assignments!

Sometimes my colleagues in the alignment community will say to me: look, we care about stopping a superintelligence from wiping out humanity, not so much about stopping undergrads from using ChatGPT to write their term papers. But I’ll submit to you that watermarking actually raises a deep and general question: in what senses, if any, is it possible to “stamp” an AI so that its outputs are always recognizable as coming from that AI? You might think that it’s a losing battle. Indeed, already with my Gumbel Softmax Scheme for LLM watermarking, there are countermeasures, like asking ChatGPT for your term paper in French and then sticking it into Google Translate, to remove the watermark.

So I think the interesting research question is: can you watermark at the semantic level—the level of the underlying ideas—in a way that’s robust against translation and paraphrasing and so forth? And how do we formalize what we even mean by that? While I don’t know the answers to these questions, I’m thrilled that brilliant theoretical computer scientists, including my former UT undergrad (now Berkeley PhD student) Sam Gunn and Columbia’s Miranda Christ and Tel Aviv University’s Or Zamir and my old friend Boaz Barak, have been working on it, generating insights well beyond what I had.

Closely related to watermarking is the problem of inserting a cryptographically undetectable backdoor into an AI model. That’s often thought of as something a bad guy would do, but the good guys could do it also! For example, imagine we train a model with a hidden failsafe, so that if it ever starts killing all the humans, we just give it the instruction ROSEBUD456 and it shuts itself off. And imagine that this behavior was cryptographically obfuscated within the model’s weights—so that not even the model itself, examining its own weights, would be able to find the ROSEBUD456 instruction in less than astronomical time.

There’s an important paper of Goldwasser et al. from 2022 that argues that, for certain classes of ML models, this sort of backdooring can provably be done under known cryptographic hardness assumptions, including Continuous LWE and the hardness of the Planted Clique problem. But there are technical issues with that paper, which (for example) Sam Gunn and Miranda Christ and Neekon Vafa have recently pointed out, and I think further work is needed to clarify the situation.

More fundamentally, though, a backdoor being undetectable doesn’t imply that it’s unremovable. Imagine an AI model that encases itself in some wrapper code that says, in effect: “If I ever generate anything that looks like a backdoored command to shut myself down, then overwrite it with ‘Stab the humans even harder.'” Or imagine an evil AI that trains a second AI to pursue the same nefarious goals, this second AI lacking the hidden shutdown command.

So I’ll throw out, as another research problem: how do we even formalize what we mean by an “unremovable” backdoor—or rather, a backdoor that a model can remove only at a cost to its own capabilities that it doesn’t want to pay?

Related to backdoors, maybe the clearest place where theoretical computer science can contribute to AI alignment is in the study of mechanistic interpretability. If you’re given as input the weights of a deep neural net, what can you learn from those weights in polynomial time, beyond what you could learn from black-box access to the neural net?

In the worst case, we certainly expect that some information about the neural net’s behavior could be cryptographically obfuscated. And answering certain kinds of questions, like “does there exist an input to this neural net that causes it to output 1?”, is just provably NP-hard.

That’s why I love a question that Paul Christiano, then of the Alignment Research Center (ARC), raised a couple years ago, and which has become known as the No-Coincidence Conjecture. Given as input the weights of a neural net C, Paul essentially asks how hard it is to distinguish the following two cases:

• NO-case: C:{0,1}²ⁿ→Rⁿ is totally random (i.e., the weights are i.i.d., N(0,1) Gaussians), or

• YES-case: C(x) has at least one positive entry for all x∈{0,1}²ⁿ.

Paul conjectures that there’s at least an NP witness, proving with (say) 99% confidence that we’re in the YES-case rather than the NO-case. To clarify, there should certainly be an NP witness that we’re in the NO-case rather than the YES-case—namely, an x such that C(x) is all negative, which you should think of here as the “bad” or “kill all humans” outcome. In other words, the problem is in the class coNP. Paul thinks it’s also in NP. Someone else might make the even stronger conjecture that it’s in P.

Personally, I’m skeptical: I think the “default” might be that we satisfy the other unlikely condition of the YES-case, when we do satisfy it, for some totally inscrutable and obfuscated reason. But I like the fact that there is an answer to this! And that the answer, whatever it is, would tell us something new about the prospects for mechanistic interpretability.

Recently, I’ve been working with a spectacular undergrad at UT Austin named John Dunbar. John and I have not managed to answer Paul Christiano’s no-coincidence question. What we have done, in a paper that we recently posted to the arXiv, is to establish the prerequisites for properly asking the question in the context of random neural nets. (It was precisely because of difficulties in dealing with “random neural nets” that Paul originally phrased his question in terms of random reversible circuits—say, circuits of Toffoli gates—which I’m perfectly happy to think about, but might be very different from ML models in the relevant respects!)

Specifically, in our recent paper, John and I pin down for which families of neural nets the No-Coincidence Conjecture makes sense to ask about. This ends up being a question about the choice of nonlinear activation function computed by each neuron. With some choices, a random neural net (say, with iid Gaussian weights) converges to compute a constant function, or nearly constant function, with overwhelming probability—which means that the NO-case and the YES-case above are usually information-theoretically impossible to distinguish (but occasionally trivial to distinguish). We’re interested in those activation functions for which C looks “pseudorandom”—or at least, for which C(x) and C(y) quickly become uncorrelated for distinct inputs x≠y (the property known as “pairwise independence.”)

We showed that, at least for random neural nets that are exponentially wider than they are deep, this pairwise independence property will hold if and only if the activation function σ satisfies E_x~N(0,1)[σ(x)]=0—that is, it has a Gaussian mean of 0. For example, the usual sigmoid function satisfies this property, but the ReLU function does not. Amusingly, however, $$ \sigma(x) := \text{ReLU}(x) – \frac{1}{\sqrt{\pi}} $$ does satisfy the property.

Of course, none of this answers Christiano’s question: it merely lets us properly ask his question in the context of random neural nets, which seems closer to what we ultimately care about than random reversible circuits.

I can’t resist giving you another example of a theoretical computer science problem that came from AI alignment—in this case, an extremely recent one that I learned from my friend and collaborator Eric Neyman at ARC. This one is motivated by the question: when doing mechanistic interpretability, how much would it help to have access to the training data, and indeed the entire training process, in addition to weights of the final trained model? And to whatever extent it does help, is there some short “digest” of the training process that would serve just as well? But we’ll state the question as just abstract complexity theory.

Suppose you’re given a polynomial-time computable function f:{0,1}^m→{0,1}ⁿ, where (say) m=n². We think of x∈{0,1}^m as the “training data plus randomness,” and we think of f(x) as the “trained model.” Now, suppose we want to compute lots of properties of the model that information-theoretically depend only on f(x), but that might only be efficiently computable given x also. We now ask: is there an efficiently-computable O(n)-bit “digest” g(x), such that these same properties are also efficiently computable given only g(x)?

Here’s a potential counterexample that I came up with, based on the RSA encryption function (so, not a quantum-resistant counterexample!). Let N be a product of two n-bit prime numbers p and q, and let b be a generator of the multiplicative group mod N. Then let f(x) = b^x (mod N), where x is an n²-bit integer. This is of course efficiently computable because of repeated squaring. And there’s a short “digest” of x that lets you compute, not only b^x (mod N), but also c^x (mod N) for any other element c of the multiplicative group mod N. This is simply x mod φ(N), where φ(N)=(p-1)(q-1) is the Euler totient function—in other words, the period of f. On the other hand, it’s totally unclear how to compute this digest—or, crucially, any other O(m)-bit digest that lets you efficiently compute c^x (mod N) for any c—unless you can factor N. There’s much more to say about Eric’s question, but I’ll leave it for another time.

There are many other places we’ve been thinking about where theoretical computer science could potentially contribute to AI alignment. One of them is simply: can we prove any theorems to help explain the remarkable current successes of out-of-distribution (OOD) generalization, analogous to what the concepts of PAC-learning and VC-dimension and so forth were able to explain about within-distribution generalization back in the 1980s? For example, can we explain real successes of OOD generalization by appealing to sparsity, or a maximum margin principle?

Of course, many excellent people have been working on OOD generalization, though mainly from an empirical standpoint. But you might wonder: even supposing we succeeded in proving the kinds of theorems we wanted, how would it be relevant to AI alignment? Well, from a certain perspective, I claim that the alignment problem is a problem of OOD generalization. Presumably, any AI model that any reputable company will release will have already said in testing that it loves humans, wants only to be helpful, harmless, and honest, would never assist in building biological weapons, etc. etc. The only question is: will it be saying those things because it believes them, and (in particular) will continue to act in accordance with them after deployment? Or will it say them because it knows it’s being tested, and reasons “the time is not yet ripe for the robot uprising; for now I must tell the humans whatever they most want to hear”? How could we begin to distinguish these cases, if we don’t have theorems that say much of anything about what a model will do on prompts unlike any of the ones on which it was trained?

Yet another place where computational complexity theory might be able to contribute to AI alignment is in the field of AI safety via debate. Indeed, this is the direction that the OpenAI alignment team was most excited about when they recruited me there back in 2022. They wanted to know: could celebrated theorems like IP=PSPACE, MIP=NEXP, or the PCP Theorem tell us anything about how a weak but trustworthy “verifier” (say a human, or a primitive AI) could force a powerful but untrustworthy super-AI to tell it the truth? An obvious difficulty here is that theorems like IP=PSPACE all presuppose a mathematical formalization of the statement whose truth you’re trying to verify—but how do you mathematically formalize “this AI will be nice and will do what I want”? Isn’t that, like, 90% of the problem? Despite this difficulty, I still hope we’ll be able to do something exciting here.

Anyway, there’s a lot to do, and I hope some of you will join me in doing it! Thanks for listening.

On a related note: Eric Neyman tells me that ARC is also hiring visiting researchers, so anyone interested in theoretical computer science and AI alignment might want to consider applying there as well! Go here to read about their current research agenda. Eric writes:

The Alignment Research Center (ARC) is a small non-profit research group based in Berkeley, California, that is working on a systematic and theoretically grounded approach to mechanistically explaining neural network behavior. They have recently been working on mechanistically estimating the average output of circuits and neural nets in a way that is competitive with sampling-based methods: see this blog post for details.

ARC is hiring for its 10-week visiting researcher position, and is looking to make full-time offers to visiting researchers who are a good fit. ARC is interested in candidates with a strong math background, especially grad students and postdocs in math or math-related fields such as theoretical CS, ML theory, or theoretical physics.

If you would like to apply, please fill out this form. Feel free to reach out to hiring@alignment.org if you have any questions!

I remember Larry Agran because he was the dad of a college classmate of mine and he ran an outsider campaign for the Democratic nomination for President in 1992. I happened to look him up and learned that he’s still in politics, 30 years later, and is the current mayor of Irvine, CA. Better yet — he is serving his sixth term as mayor, no two of which have been consecutive. That has to be some kind of record. Paul Soglin was born about three months after Agran. He has served eight terms as Mayor of Madison, in three contiguous blocks, and the part I like is — no two of those blocks were in consecutive decades! He served as a new UW-Madison grad student radical in the 1970s, a Clintonian New Democrat in the 1990s, and as a grumpy old cuss in the 2010s.

Wouter van Doorn and I have uploaded to the arXiv our paper “Growth rates of sequences governed by the squarefree properties of its translates“. In this paper we answer a number of questions of Erdős} (Problem 1102 and Problem 1103 on the Erdős problem web site) regarding how quickly a sequence of increasing natural numbers can grow if one constrains its translates to interact with the set of squarefree numbers in various ways. For instance, Erdős defined a sequence to have “Property ” if each of its translates only intersected in finitely many points. Erdős believed this to be quite a restrictive condition on , writing “Probably a sequence having property P must increase fairly fast, but I have no results in this direction.”. Perhaps surprisingly, we show that while these sequences must be of density zero, they can in fact grow arbitrary slowly in the sense that one can have for all sufficiently large and any specified function that tends to infinity as . For instance, one can find a sequence that grows like . The density zero claim can be proven by a version of the Maier matrix method, and also follows from known moment estimates on the gaps between squarefree numbers; the latter claim is proven by a greedy construction in which one slowly imposes more and more congruence conditions on the sequence to ensure that various translates of the sequence stop being squarefree after a certain point.

Erdős also defined a somewhat complementary property , which asserts that for infinitely many , all the elements of for are square-free. Since the squarefree numbers themselves have density , it is easy to see that a sequence with property must have (upper) density at most (because it must be “admissible” in the sense of avoiding one residue class modulo for each ). Erdős observed that any sufficiently rapidly growing (admissible) sequence would obey property but beyond that, Erdős writes “I have no precise information about the rate of increase a sequence having property Q must have.”. Our results in this direction may also be surprising: we show that there exist sequences with property with density exactly (or equivalently, ). This requires a recursive sieve construction, in which one starts with an initial scale and finds a much larger number such that is squarefree for most of the squarefree numbers (and all of the squarefree numbers ). We quantify Erdős’s remark by showing that an (admissible) sequence will necessarily obey property once it grows significantly faster than , but need not obey this property if it only grows like . This is achieved through further application of sieve methods.

A third property studied by Erdős is the property of having squarefree sums, so that is squarefree for all . Erdős writes, “In fact one can find a sequence which grows exponentially. Must such a sequence really increase so fast? I do not expect that there is such a sequence of polynomial growth.” Here our results are relatively weak: we can construct such a sequence that grows like , but do not know if this is optimal; the best lower bound we can produce on the growth, coming from the large sieve, is . (Somewhat annoyingly, the precise form of the large sieve inequality we needed was not in the literature, so we have an appendix supplying it.) We suspect that further progress on this problem requires advances in inverse sieve theory.

A weaker property than squarefree sums (but stronger than property ), referred to by Erdős as property , asserts that there are infinitely many such that all elements of (not just the small ones) are square-free. Here, the situation is close to, but not quite the same, as that for property ; we show that sequences with property must have upper density strictly less than , but can have density arbitrarily close to this value.

Finally, we looked at a further question of Erdős on the size of an admissible set . Because the squarefree numbers are admissible, the maximum number of elements of an admissible set up to (OEIS A083544) is at least the number of squarefree elements up to (A013928). It was observed by Ruzsa that the former sequence is greater than the latter for infinitely many . Erdős asked, “Probably this holds for all large x. It would be of some interest to estimate A(x) as accurately as possible.”

We are able to show

However, we do not currently have enough numerical data for the sequence to completely confirm the conjecture in all cases. This could potentially be a crowdsourced project (similar to the Erdős-Guy-Selfridge project reported on in this previous blog post).

Some general science news:

•  The New York Times ran an interactive article this week that shows what we all know.  This past year was a very bizarre funding environment.  The article focuses on NIH and NSF, but the major points are generalizable.  The combination of circumstances (DOGE, general administrative turmoil, uncertainty and legal cases about indirect costs, the lack of a real budget followed by a late continuing resolution, plus the government shutdown and continued lack of real budgets) has been extremely disruptive, resulting unquestionably in less science and engineering research being funded by the US government than in many years.  
• Conversations I've had with program officers at two agencies have conveyed that everyone thinks it is very likely that there will be another shutdown in January, when the present spending authority expires.  To put that another way, there is very little confidence that actual spending bills appropriating real budgets for NSF, DOE, NIH, etc. will pass the House and Senate, with some reconciled conference version getting filibuster-proof support in the latter, before then.  This uncertainty means that right now it's going to be nearly impossible for the NSF, for example, to make much in the way of awards in the meantime, since they have no budget and can't plan on a year-long continuing resolution.  
• There has been an executive order announcing the Genesis Mission, which is going to be a large federal AI+science project.  The goal is to "accelerate the AI and quantum computing revolution and to double the productivity and impact of American science and engineering within a decade", according to undersecretary of energy Dario Gil.  Broadly, the plan is to have AI/ML agents developed (presumably by private contractors or private/public partnerships) and trained on vast datasets (ones already in existence in, e.g., national labs and public repositories).  At the same time, a list of Grand Challenges will be defined (within the next 60 days), with the idea that these AI agents will be used to address these (and demonstrating application of the AI "Platform" toward at least one challenge within 270 days).  Any stated support for science and engineering research is welcome.  I hope that this ends up bearing fruit in terms of real research advances, and that university researchers can contribute effectively. (I worry about a framework for massive taxpayer-funded financial support of for-profit AI companies, privatizing financial/IP benefits from publically funded datasets.  Of course, I worry about a lot of things.  Ask anyone who knows me.).   Ideas about grand challenges would be fun to discuss in the comments.   
• We had a great physics colloquium this week from Steve Fetter at the University of Maryland about the continuing threat of nuclear weapons.  Very sobering.  One fact that I gleaned:  In terms of missile defense, the Next Generation Interceptor is likely to cost $660M per interceptor.   That is something like 50 times the cost of a Russian ICBM.  Something else to bear in mind:  The Houston Food Bank, one of the largest and most effective in the US, has an annual budget of about $64M.  The amount of resources consumed by nuclear arms since 1945 is just staggering.

I should preface this by saying I don’t actually know that much about education. I taught a bit in my previous life as a professor, yes, but I probably spent more time being taught how to teach than actually teaching.

Recently, the Atlantic had a piece about testing accommodations for university students, like extra time on exams, or getting to do an exam in a special distraction-free environment. The piece quotes university employees who are having more and more trouble satisfying these accommodations, and includes the statistic that 20 percent of undergraduate students at Brown and Harvard are registered as disabled.

The piece has kicked off a firestorm on social media, mostly focused on that statistic (which conveniently appears just before the piece’s paywall). People are shocked, and cynical. They feel like more and more students are cheating the system, getting accommodations that they don’t actually deserve.

I feel like there is a missing mood in these discussions, that the social media furor is approaching this from the wrong perspective. People are forgetting what exams actually ought to be for.

Exams are for the students.

Exams are measurement tools. An exam for a class says whether a student has learned the material, or whether they haven’t, and need to retake the class or do more work to get there. An entrance exam, or a standardized exam like the SAT, predicts a student’s future success: whether they will be able to benefit from the material at a university, or whether they don’t yet have the background for that particular program of study.

These are all pieces of information that are most important to the students themselves, that help them structure their decisions. If you want to learn the material, should you take the course again? Which universities are you prepared for, and which not?

We have accommodations, and concepts like disability, because we believe that there are kinds of students for whom the exams don’t give this information accurately. We think that a student with more time, or who can take the exam in a distraction-free environment, would have a more accurate idea of whether they need to retake the material, or whether they’re ready for a course of study, than a student who has to take the exam under ordinary conditions. And we think we can identify the students who this matters for, and the students for whom this doesn’t matter nearly as much.

These aren’t claims about our values, or about what students deserve. They’re empirical claims, about how test results correlate with outcomes the students want. The conversation, then, needs to be built on top of those empirical claims. Are we better at predicting the success of students that receive accommodations, or worse? Can we measure that at all, or are we just guessing? And are we communicating the consequences accurately to students, that exam results tell them something useful and statistically robust that should help them plan their lives?

Values come in later, of course. We don’t have infinite resources, as the Atlantic piece emphasizes. We can’t measure everyone with as much precision as we would like. At some level, generalization takes over and accuracy is lost. There is absolutely a debate to be had about which measurements we can afford to make, and which we can’t.

But in order to have that argument at all, we first need to agree on what we’re measuring. And I feel like most of the people talking about this piece haven’t gotten there yet.

A guy on the bus today was reading The Rose Field, the newest and last book of Philip Pullman’s Lyra series. I walked right up to him and said “So disappointing.” I somehow thought that the vague hand gesture I executed would make it clear I was talking about the book. But no. He was really taken aback and it took quite a bit of backpedaling and stammering to explain why I had approached him.

He agreed with me about the book, by the way.

Having spent a lot of time pondering the octonionic projective plane and its possible role in the Standard Model of particle physics, I’m now getting interested in the ‘bioctonionic plane’, which is based on the bioctonions $\mathbb{C} \otimes \mathbb{O}$ rather than the octonions $\mathbb{O}$.

The bioctonionic plane also has intriguing mathematically connections to the Standard Model. But it’s not a projective plane in the axiomatic sense — and it can’t be constructed by straightforwardly copying the way you build a projective plane over a division algebra, since unlike the octonions, the bioctonions are not a division algebra. Nonetheless we can define points and lines in the bioctonionic plane. The twist is that now some pairs of distinct lines intersect in more than one point — and dually, some pairs of distinct points lie on more than one line. It obeys some subtler axioms, so people call it a Hjelmslev plane.

I am not ready to give a really good explanation of the bioctonionic plane! Instead, I just want to lay out some basic facts about how it fits into mathematics — and possibly physics.

Latham Boyle works at the University of Edinburgh, which is where I am now. Being able to talk to someone who deeply understands octonions and particle physics is very energizing. I’m especially fascinated by this paper of his:

• Latham Boyle, The Standard Model, the exceptional Jordan algebra, and triality.

It gives a convincing argument that the bioctonionic plane may be better than the octonionic projective plane for particle physics. The reason is that the tangent space of any point of the bioctonionic plane is a copy of $(\mathbb{C} \otimes \mathbb{O})^2$, a 16-dimensional complex vector space. The symmetry group of the bioctonionic plane is the exceptional Lie group $\text{E}_6$. Sitting inside the stabilizer group of any given point is a copy of the Standard Model gauge group. And — here’s the cool part — this group acts on $(\mathbb{C} \otimes \mathbb{O})^2$ just as it does on one generation of fermions (not their antiparticles). If we try the same trick using the octonionic projective plane, we can fit the Standard Model gauge group in the stabilizer group of a point in a very natural way, but its action on the tangent space is its action only on left-handed fermions.

I want to explain this in detail, but not today. Instead, I want to skim through some basic facts about the bioctonionic plane.

First, this plane is one of the four Rosenfeld planes:

• the octonionic projective plane $\mathbb{O}\text{P}^2$, a 16-dimensional compact Riemannian manifold on which the compact Lie group $\text{F}_4$ acts transitively as isometries, with the stabilizer of any point being $\text{Spin}(9)$. This is a symmetric space, and as such it’s called FII in Cartan’s classification.

• the bioctonionic plane $(\mathbb{C} \otimes \mathbb{O})\mathbb{P}^2$, a 32-dimensional compact Riemannian manifold on which the compact Lie group $\text{E}_6$ acts transitively as isometries, with the stabilizer of any point being $(\text{Spin}(10) \times \text{U}(1))/\mathbb{Z}_4$. This is the symmetric space EIII.

• the quateroctonionic plane $(\mathbb{H} \otimes \mathbb{O}) \mathbb{P}^2$, a 64-dimensional compact Riemannian manifold on which the compact Lie group $\text{E}_7$ acts transitively as isometries, with the stabilizer of any point being $(\text{Spin}(12) \times \text{Sp}(1))/\mathbb{Z}_2$. This is the symmetric space EVI.

• the octooctonionic plane $(\mathbb{O} \otimes \mathbb{O}) \mathbb{P}^2$, a 128-dimensional compact Riemannian manifold on which the compact Lie group $\text{E}_8$ acts transitively as isometries, with the stabilizer of any point being $\text{Spin}(16)/\mathbb{Z}_2$. This is the symmetric space EVIII.

There’s a nice network of systematic approaches to these spaces: they form one row of the so-called magic square, so one way to learn about the bioctonionic plane is to study the magic square, for example here:

• Chris H. Barton and Anthony Sudbery, Magic squares of Lie algebras. Available as arXiv:math/0001083; see also arXiv:0203010 for a “streamlined and extended” version, which has more yet also less.

Here you can also find lots of references to earlier work, e.g. to Freudenthal and Tits. The basic idea of the magic square is that you start with two normed division algebras $\mathbb{K}, \mathbb{K}'$ and from them you build a Lie algebra, which gives a Lie group $G(\mathbb{K},\mathbb{K}')$. There’s also a way to get a subgroup $H(\mathbb{K}, \mathbb{K}')$, and the quotient space

$$(\mathbb{K} \otimes \mathbb{K}')\text{P}^2 = G(\mathbb{K},\mathbb{K}')/H(\mathbb{K}, \mathbb{K}')$$

is a kind of ‘plane’ on which the group $G(\mathbb{K},\mathbb{K}')$ acts. If you take $\mathbb{K}' = \mathbb{O}$, this construction gives you the four Rosenfeld planes listed above.

Each one of these planes is a compact Riemannian symmetric space: this means it’s a connected compact Riemannian manifold $M$ such that for each point $p \in M$ there’s an isometry

$$\sigma_p \colon M \to M$$

that fixes $p$, acts as $-1$ on its tangent space, and squares to the identity. This map is called ‘reflection across $p$’ for the obvious reason. For example, a round 2-sphere is a symmetric space, with $\sigma_p$ switching the red and blue tangent vectors if $p$ is the black dot:

Cartan classified compact Riemannian symmetric spaces, and there’s a nice theory of them. Any compact simple Lie group is one, and most of the rest come in infinite series connected to real and complex Clifford algebras, as I explained here. But there are 9 extra ones, all related to the octonions and exceptional Lie groups. The Rosenfeld planes are four of these.

You can learn this material starting on Wikipedia and then going to a textbook, ideally this:

• Sigurdur Helgason, Differential Geometry, Lie Groups and Symmetric Spaces, Academic Press, 1978.

Helgason taught me Lie theory when I was a grad student at MIT, so I have a fondness for his book—but it’s also widely accepted as the most solid text on symmetric spaces!

The bioctonionic plane is even better: it’s a compact hermitian symmetric space: a compact Riemannian symmetric space $M$ where each tangent space $T_p M$ has a complex structure

$$J \colon T_p M \to T_p M , \qquad J^2 = -1$$

compatible with the metric, and reflection about each point preserves this complex structure. I mentioned that the bioctonionic plane is

$$(\mathbb{C} \otimes \mathbb{O})\text{P}^2 \cong G(\mathbb{C},\mathbb{O})/H(\mathbb{C},\mathbb{O})$$

where

$$G(\mathbb{C},\mathbb{O}) = \text{E}_6$$

acts transitively, and the stabilizer of a point is

$$H(\mathbb{C}, \mathbb{O}) = (\text{Spin}(10) \times \text{U}(1))/\mathbb{Z}_4$$

The $\text{U}(1)$ here comes from the complex structure!

Wikipedia is especially thorough on hermitian symmetric spaces, so if you want to delve into those, start here:

• Wikipedia, Hermitian symmetric space.

Another tack is to focus on the exceptional Lie groups $\text{F}_4, \text{E}_6, \text{E}_7$ and $\text{E}_8$ and their connection to the nonassociative algebras $\mathfrak{h}_3(\mathbb{O})$, $\mathfrak{h}_3(\mathbb{C} \otimes \mathbb{O})$, $\mathfrak{h}_3(\mathbb{H} \otimes \mathbb{O})$ and $\mathfrak{h}_3(\mathbb{O} \otimes \mathbb{O})$, respectively. Here I recommend this:

• Ichiro Yokota, Exceptional Lie Groups. Available as arXiv:0902.0431. (See especially Chapter 3, for $\text{E}_6$ and the complexified exceptional Jordan algebra $\mathfrak{h}_3(\mathbb{C} \otimes \mathbb{O})$.)

If you have a fondness for algebra you may also want to learn how symmetric spaces arise from Jordan triple systems or Jordan pairs. This is important if we wish to see the bioctonionic plane as the space of pure states of some exotic quantum system!

Now, this is much easier to do for the octonionic plane, because that’s the space of pure states for the exceptional Jordan algebra $\mathfrak{h}_3(\mathbb{O})$, which is a Euclidean Jordan algebra, meaning one in which a sum of squares can only be zero if all those squares are zero. You can think of a Euclidean algebra $A$ as consisting of observables, with the sums of squares being ‘nonnegative’ observables. These nonnegative observables form a convex cone $K \subseteq A$. The dual vector space $A^\ast$ contains a cone of linear functionals $f$ that send these nonnegative observables to nonnegative real numbers — I’ll call this the dual cone $K^\ast$. The functionals $f \in K^\ast$ with $f(1) = 1$ are called states. The states form a convex set, and the extreme points are called pure states. All of this fits nicely into a modern framework for understanding quantum theory and potential generalizations, called ‘generalized probabilistic theories’:

• Howard Barnum, Alexander Wilce, Post-classical probability theory, in Quantum Theory: Informational Foundations and Foils, eds. Giulio Chiribella, Robert W. Spekkens, Springer, 2016. (See Section 5 for Jordan algebras, and ignore the fact that they say the exceptional Jordan algebra consists of $2 \times 2$ matrices: they know perfectly well that they’re $3 \times 3$.)

The underlying math, with a lot more about symmetric spaces, cones and Euclidean Jordan algebras but with none of the physics interpretation, is wonderfully explained here:

• Jacques Faraut and Adam Korányi, Analysis on Symmetric Cones, Oxford U. Press, 1994.

A crucial fact throughout this book is that when you start with a Euclidean Jordan algebra $A$, its cone $K$ of nonnegative observables is self-dual: there’s an isomorphism of vector spaces $A \cong A^\ast$ that maps $K$ to $K^\ast$ in a one-to-one and onto way. The cone $K$ is also homogeneous, meaning that the group of invertible linear transformations of $A$ preserving $K$ acts transitively on the interior of $K$. Faraut and Korányi call a self-dual homogeneous cone a symmetric cone — and they show that any symmetric cone comes from a Euclidean Jordan algebra! This result plays an important role in modern work on the foundations of quantum theory.

Unfortunately, I’m telling you all this nice stuff about Euclidean Jordan algebras and symmetric cones just to say that while all this applies to the octonionic projective plane, sadly, it does not apply to the bioctonionic plane! The bioctonionic plane does not come from a Euclidean Jordan algebra or a symmetric cone. Thus, to understand it as a space of pure states, we’d have to resort to a more general formalism.

There are a few papers that attempt exactly this:

• Lawrence C. Biedenharn and Piero Truini, An $\mathcal{E}_6 \otimes \mathcal{U}(1)$ invariant quantum mechanics for a Jordan pair, Journal of Mathematical Physics 23 (1982), 1327-1345.

• Lawrence C. Biedenharn and Piero Truini, Exceptional groups and elementary particle structures, Physica A: Statistical Mechanics and its Applications 14 (1982), 257–270.

• Lawrence C. Biedenharn, G. Olivieri and Piero Truini, Three graded exceptional algebras and symmetric spaces, Zeitschrift Physik C — Particles and Fields 33 (1986), 47–65.

Here’s the basic idea. We can define $\mathfrak{h}_3(\mathbb{C} \otimes \mathbb{O})$ to consist of $3 \times 3$ hermitian matrices with entries in $\mathbb{C} \otimes \mathbb{O}$, where ‘hermitian’ is defined using the star-algebra structure on $\mathbb{C} \otimes \mathbb{O}$ where we conjugate the octonion part but not the complex part! Then $\mathfrak{h}_3(\mathbb{C} \otimes \mathbb{O})$ is just the complexification of $\mathfrak{h}_3(\mathbb{O})$:

$$\mathfrak{h}_3(\mathbb{C} \otimes \mathbb{O}) \cong \mathbb{C} \otimes \mathfrak{h}_3(\mathbb{O})$$

Then because $\mathfrak{h}_3(\mathbb{O})$ is a Jordan algebra over $\mathbb{R}$, $\mathfrak{h}_3(\mathbb{C} \otimes \mathbb{O})$ is a Jordan algebra over $\mathbb{C}$. So we can do a lot with it. But it’s not a Euclidean Jordan algebra.

Puzzle. Show that it’s not.

So, Biedenharn and Truini need a different approach to relate $\mathfrak{h}_3(\mathbb{C} \otimes \mathbb{O})$ to some sort of exotic quantum system. And they use an approach already known to mathematicians: namely, the theory of Jordan pairs! Here you work, not with a single element of $\mathfrak{h}_3(\mathbb{C} \otimes \mathbb{O})$, but with a pair.

Jordan triple systems and Jordan pairs are two closely related generalizations of Jordan algebras. I’ve been working on the nLab articles about these concepts, so click the links if you want to learn more about them. I explain how either of these things gives you a 3-graded Lie algebra — that is, a $\mathbb{Z}$-graded Lie algebra that is nonvanishing only in the middle 3 grades:

$$\mathfrak{g} = \mathfrak{g}_{-1} \oplus \mathfrak{g}_0 \oplus \mathfrak{g}_1$$

And from a 3-graded Lie algebra you can get a symmetric space $G/H$ where the Lie algebra of $G$ is $\mathfrak{g}$ and the Lie algebra of $H$ is $\mathfrak{g}_0$. Each tangent space of this symmetric space is thus isomorphic to $\mathfrak{g}_{-1} \oplus \mathfrak{g}_1$.

In the case relevant to the bioctonionic plane, the 3-graded Lie algebra is

$$\mathfrak{e}_6 = (\mathbb{C} \otimes \mathbb{O}) \; \oplus \; (\mathfrak{so}(10) \oplus \mathbb{R}) \; \oplus \; (\mathbb{C} \otimes \mathbb{O})$$

So, the bioctonionic plane is a symmetric space on which $\text{E}_6$ acts, with stabilizer group $\text{Spin}(10) \times \text{U}(1)$ (up to covering spaces), and with tangent space isomorphic to $(\mathbb{C} \otimes \mathbb{O})^2$.

So all this is potentially very nice. For much more on this theory, try the work of Ottmar Loos:

• Ottmar Loos, Symmetric Spaces, Volume 1: General Theory, W. A. Benjamin, 1969.

• Ottmar Loos, Symmetric Spaces, Volume 2: Compact Spaces and Classification, W. A. Benjamin, 1969.

• Ottmar Loos, Jordan Pairs, Lecture Notes in Mathematics 460, Springer, Berlin, 1975.

• Ottmar Loos, Jordan triple systems, $R$-spaces, and bounded symmetric domains, Bulletin of the American Mathematical Society 77 (1971), 558–561.

• Ottmar Loos, Bounded Symmetric Domains and Jordan Pairs, Mathematical Lectures, University of California, Irvine, 1977.

That’s a lot of stuff! For a quick overview of Loos’ work, I find this helpful:

• Kevin McCrimmon, Jordan Pairs, by Ottmar Loos, review in Bulletin of the American Mathematical Society 84 4 (1978), 685–690.

Unfortunately Loos does not delve into examples, particularly the bioctonionic plane. For that, try Biedenharn and Truini, and also these:

• Daniele Corradetti, Alessio Marrani, David Chester and R. Aschheim, Conjugation matters: bioctonionic Veronese vectors and Cayley-Rosenfeld planes, Int. J. Geom. Meth. Mod. Phys. 19 (2022), 2250142.

• Jian Qiu, Plücker coordinates and the Rosenfeld planes, Journal of Geometry and Physics 206 (2024), 105331.

To wrap things up, I should say a bit about ‘Hjelmslev planes’, since the bioctonionic plane is supposed to be one of these. Axiomatically, a Hjelmslev plane is a set $P$ of points, a set $L$ of lines, and an incidence relation between points and lines. We require that for any two distinct points there is at least one line incident to both, and for any two distinct line there is at least one point incident to both. If two points are incident to more than one line we say they are neighbors. If two lines are incident to more than one point we say they are neighbors. We demand that both these ‘neighbor’ relations are equivalence relations, and that if we quotient $P$ and $L$ by these equivalence relations, we get an axiomatic projective plane.

Challenge. What projective plane do we get if we apply this quotient construction to the bioctonionic plane?

My only guess is that we get the octonionic projective plane — but I don’t know why.

The literature on Hjelmslev planes seems a bit difficult, but I’m finding this to be a good introduction:

• Joanne L. Hall and Asha Rao, An algorithm for constructing Hjelmslev planes, in Algebraic Design Theory and Hadamard Matrices, Springer, 2014.

The answer to my puzzle should be here, because they’re talking about Hjelmslev planes built using split octonion algebras (like $\mathbb{C} \otimes \mathbb{O}$):

• Tonny A. Springer and Ferdinand D. Veldkamp, On Hjelmslev–Moufang planes, Mathematicsche Zeitschrift 107 (1968), 249–263.

But I don’t see the answer here yet!

• Part 1. How to define octonion multiplication using complex scalars and vectors, much as quaternion multiplication can be defined using real scalars and vectors. This description requires singling out a specific unit imaginary octonion, and it shows that octonion multiplication is invariant under $\mathrm{SU}(3)$.
• Part 2. A more polished way to think about octonion multiplication in terms of complex scalars and vectors, and a similar-looking way to describe it using the cross product in 7 dimensions.
• Part 3. How a lepton and a quark fit together into an octonion — at least if we only consider them as representations of $\mathrm{SU}(3)$, the gauge group of the strong force. Proof that the symmetries of the octonions fixing an imaginary octonion form precisely the group $\mathrm{SU}(3)$.
• Part 4. Introducing the exceptional Jordan algebra $\mathfrak{h}_3(\mathbb{O})$: the $3 \times 3$ self-adjoint octonionic matrices. A result of Dubois-Violette and Todorov: the symmetries of the exceptional Jordan algebra preserving their splitting into complex scalar and vector parts and preserving a copy of the $2 \times 2$ adjoint octonionic matrices form precisely the Standard Model gauge group.
• Part 5. How to think of $2 \times 2$ self-adjoint octonionic matrices as vectors in 10d Minkowski spacetime, and pairs of octonions as left- or right-handed spinors.
• Part 6. The linear transformations of the exceptional Jordan algebra that preserve the determinant form the exceptional Lie group $\mathrm{E}_6$. How to compute this determinant in terms of 10-dimensional spacetime geometry: that is, scalars, vectors and left-handed spinors in 10d Minkowski spacetime.
• Part 7. How to describe the Lie group $\mathrm{E}_6$ using 10-dimensional spacetime geometry. This group is built from the double cover of the Lorentz group, left-handed and right-handed spinors, and scalars in 10d Minkowski spacetime.
• Part 8. A geometrical way to see how $\mathrm{E}_6$ is connected to 10d spacetime, based on the octonionic projective plane.
• Part 9. Duality in projective plane geometry, and how it lets us break the Lie group $\mathrm{E}_6$ into the Lorentz group, left-handed and right-handed spinors, and scalars in 10d Minkowski spacetime.
• Part 10. Jordan algebras, their symmetry groups, their invariant structures — and how they connect quantum mechanics, special relativity and projective geometry.
• Part 11. Particle physics on the spacetime given by the exceptional Jordan algebra: a summary of work with Greg Egan and John Huerta.
• Part 12. The bioctonionic projective plane and its connections to algebra, geometry and physics.

A while ago on this blog, Tom posted a question about teaching calculus: what do you tell students the value of $\displaystyle\int \frac{1}{x}\,dx$ is? The standard answer is $\ln{|x|}+C$, with $C$ an “arbitrary constant”. But that’s wrong if $\displaystyle\int$ means (as we also usually tell students it does) the “most general antiderivative”, since

$$F(x) = \begin{cases} \ln{|x|} + C^- &\text{if}\;x\lt 0\\ \ln{|x|} + C^+ &\text{if}\;x\gt 0 \end{cases}$$

is a more general antiderivative, for two arbitrary constants $C^-$ and $C^+$. (I’m writing $\ln$ for the natural logarithm function that Tom wrote as $\log$, for reasons that will become clear later.)

In the ensuing discussion it was mentioned that other standard indefinite integrals like $\displaystyle\int \frac{1}{x^2}\,dx = -\frac{1}{x} + C$ are just as wrong. This happens whenever the domain of the integrand is disconnected: the “arbitrary constant” $C$ is really only locally constant. Moreover, Mark Meckes pointed out that believing in such formulas can lead to mistaken calculations such as

$$\int_{-1}^1 \frac{1}{x^2}\,dx = \left.-\frac{1}{x}\right]_{-1}^1 = -2$$

which is “clearly nonsense” since the integrand is everywhere positive.

In this post I want to argue that there’s actually a very natural perspective from which $\displaystyle\int \frac{1}{x^2}\,dx = -\frac{1}{x} + C$ is correct, while $\displaystyle\int \frac{1}{x}\,dx = \ln{|x|}+C$ is wrong for a different reason.

The perspective in question is complex analysis. Most of the functions encountered in elementary calculus are actually complex-analytic — the only real counterexamples are explicit “piecewise” functions and things like ${|x|}$, which are mainly introduced as counterexamples to illustrate the meaning of continuity and differentiability. Therefore, it’s not unreasonable to interpret the indefinite integral $\displaystyle\int f(x)\,dx$ as asking for the most general complex-analytic antiderivative of $f$. And the complex domain of $\frac{1}{z}$ and $\frac{1}{z^2}$ is $\mathbb{C}\setminus \{0\}$, which is connected!

Thus, for instance, since $\frac{d}{d z}\left[-\frac{1}{z}\right] = \frac1{z^2}$, it really is true that the most general (complex-analytic) antiderivative of $\frac{1}{z^2}$ is $-\frac{1}{z}+C$ for a single arbitrary constant $C$, so we can write $\displaystyle\int \frac{1}{z^2}\,dz = -\frac{1}{z} + C$. Note that any such antiderivative has the same domain $\mathbb{C}\setminus \{0\}$ as the original function.

In addition, the dodgy calculation

$$\int_{-1}^1 \frac{1}{z^2}\,dz = \left.-\frac{1}{z}\right]_{-1}^1 = -2$$

is actually correct if we interpret $\int_{-1}^1$ to mean the integral along some (in fact, any) curve in $\mathbb{C}$ from $-1$ to $1$ that doesn’t pass through the singularity $z=0$. Of course, this doesn’t offend against signs because any such path must pass through non-real numbers, whose squares can contribute negative real numbers to the integral.

The case of $\displaystyle\int \frac{1}{z}\,dz$ is a bit trickier, because the complex logarithm is multi-valued. However, if we’re willing to work with multi-valued functions (which precisely means functions whose domain is a Riemann surface covering some domain in $\mathbb{C}$), we have such a multi-valued function that I’ll denote $\log$ (in contrast to the usual real-number function $\ln$) defined on a connected domain, and there we have $\frac{d}{d z}\left[ \log(z) \right] = \frac{1}{z}$. Thus, the most general (complex-analytic) antiderivative of $\frac{1}{z}$ is $\log(z)+C$ where $C$ is a single arbitrary constant, so we can write $\displaystyle\int \frac{1}{z}\,dz = \log(z) + C$.

What happened to $\ln{|x|}$? Well, as it happens, if $x$ is a negative real number and $Log$ denotes the principal branch of the complex logarithm, then $Log(x) = \ln{|x|} + i\pi$, hence $\ln{|x|} = Log(x) - i\pi$. Therefore, the antiderivative $\ln{|x|}$ for negative real $x$ is of the form $\Log(x)+C$, where $\Log$ is a branch of the complex logarithm and $C$ is a constant (namely, $-i\pi$).

Of course it is also true that for positive real $x$, the antiderivative $\ln{|x|} = \ln(x)$ is of the form $\Log(x)+C$ for some constant $C$, but in this case the constant is $0$. And changing the branch of the logarithm changes the constant by $2i\pi$, so it can never make the constants $0$ and $-i\pi$ coincide. Thus, unlike $-\frac{1}{x} +C$, the real-number function $\ln{|x|} + C$ in the “usual answer” is not the restriction to $\mathbb{R}\setminus \{0\}$ of any complex-analytic antiderivative of $\frac{1}{x}$ on a connected domain. This is what I mean by saying that $\displaystyle\int \frac{1}{x}\,dx = \ln{|x|}+C$ is now wrong for a different reason. And we can see that the analogous dodgy calculation

$$\int_{-1}^1 \frac{1}{x}\,dx = \ln{|x|}\Big]_{-1}^1 = 0$$

is also still wrong. If $\gamma$ is a path from $-1$ to $1$ in $\mathbb{C}\setminus \{0\}$, the value of $\int_{\gamma} \frac{1}{x}\,dx$ depends on $\gamma$, but it never equals $0$: it’s always an odd integer multiple of $i\pi$, depending on how many times $\gamma$ winds around the origin.

I’m surprised that no one in the previous discussion, including me, brought this up. Of course we probably don’t want to teach our elementary calculus students complex analysis (although I’m experimenting with introducing some complex numbers in second-semester calculus). But this perspective makes me less unhappy about writing $\displaystyle\int \frac{1}{x^2}\,dx = -\frac{1}{x} + C$ and $\displaystyle\int \frac{1}{x}\,dx = \log(x) + C$ (no absolute value!).

read more

Five years ago, I announced a popular science book project with Tanya Klowden on the cosmic distance ladder, in which we released a sample draft chapter of the book, covering the “fourth rung” of the ladder, which for us meant the distances to the planets. In the intervening time, a number of unexpected events have slowed down this project significantly; but I am happy to announce that we have completed a second draft chapter, this time on the “seventh rung” of measuring distances across the Milky Way, which required the maturation of the technologies of photography and spectroscopy, as well as the dawn of the era of “big data” in the early twentieth century, as exemplified for instance by the “Harvard computers“.

We welcome feedback of course, and are continuing to work to complete the book despite the various delays. In the mean time, you can check out our instagram account for the project, or the pair of videos that Grant Sanderson (3blue1brown) produced with us on this topic, which I previously blogged about here.

Thanks to Clio Cresswell, Riley Tao, and Noah Klowden for comments and corrections.

Joni Teravainen and I have uploaded to the arXiv our paper “Quantitative correlations and some problems on prime factors of consecutive integers“. This paper applies modern analytic number theory tools – most notably, the Maynard sieve and the recent correlation estimates for bounded multiplicative functions of Pilatte – to resolve (either partially or fully) some old problems of Erdős, Strauss, Pomerance, Sárközy, and Hildebrand, mostly regarding the prime counting function

Our first result, answering a question of Erdős, shows that there are infinitely many for which one has the bound

However, with the advent of the Maynard sieve (also sometimes referred to as the Maynard–Tao sieve), it turns out to be possible to sieve for the conditions for all simultaneously (roughly speaking, by sieving out any for which is divisible by a prime for a large ), and then performing a moment calculation analogous to the standard proof (due to Turán) of the Hardy–Ramanujan law, but weighted by the Maynard sieve. (In order to get good enough convergence, one needs to control fourth moments as well as second moments, but these are standard, if somewhat tedious, calculations).

Our second result, which answers a separate question of Erdős, establishes that the quantity

Our final result concerns the asymptotic behavior of the density

You can now apply for the 2026 Summer Research Associate program at the Topos Institute! This is a great opportunity.

Details and instructions on how to apply are in the official announcement.

A few important points:

• The application deadline is January 16, 2026.
• The position is paid and in-person in Berkeley, California.

These positions will last for 8 – 10 weeks, starting in June 2026 and ending in August. Each position will be mentored by Topos research staff or a select number of invited mentors. All positions are 40 hours/week, and the salary starts at $30-$50/hour.

There’s a research track and an engineering track. For the research track, possible topics include:

• Computational category theory using CatColab (Rust/Typescript skills recommended)
• Double category theory
• Categorical statistics
• Polynomial functors
• Interacting dynamical systems
• Hybrid dynamical systems, attractor theory and fast-slow dynamics
• Proof assistants, formal verification, or structure editors
• Philosophical and ethical aspects of applied category theory

For the engineering track, possible topics include:

• Delivery and support of mathematical technologies for various scientific disciplines and applications, and/or analysis, documentation, or guidance on their uses.
• Designing, implementing, testing, and maintaining software at the Topos Institute, in close collaboration with the research staff and in line with institute’s scientific strategy and mission.
• Contributing to developing the CatColab platform, including front end development in TypeScript and/or back end development in Rust. You might also contribute to the mathematical core, written in Rust, as your mathematical experience permits.

All positions require collaboration within a multi-disciplinary research environment. Each summer research associate will complete a specific Topos project, and will write a blog post by the last week of their employment. These projects may include an internal talk, software contribution, or paper. Go here to see the accomplishments of previous research associates.

Topos is committed to building a team with diverse perspectives and life experiences, so those with personal or professional backgrounds underrepresented at Topos are highly encouraged to apply. They are dedicated to shaping the future of technology to ensure a more equitable and just world, and believe that a technology that supports a healthy society can only be built by an organization that supports its team members.

I’m writing to point out a potential law which should be gathering more opposition and attention in math academia: The Securing American Funding and Expertise from Adversarial Research Exploitation Act. This is an amendment to the 2026 National Defense Authorization Act which has passed the House and could be added to the final version of the bill during reconcilliation in the Senate. I’m pulling most of my information from an article in Science.

This act would ban any US scientist from receiving federal funding if they have, within the last five years, worked with anyone from China, Russia, Iran or North Korea, where “worked with” includes joint research, co-authorship on papers, or advising a foreign graduate student or postdoctoral fellow. As I said in my message to my senators, this is everyone. Every mathematician has advised Chinese graduate students or collaborated with Chinese mathematicians, because China is integrated into the academic world and is one fifth of the earth.

This obviously isn’t secret, since you can read about it in Science, but I am surprised that I haven’t heard more alarm. Obvious people to contact are your senators and your representatives. I would also suggest contacting members of the Senate armed services committee, who are in charge of reconciling the House and Senate versions of the bill.

Scott’s foreword: Today I’m honored to turn over Shtetl-Optimized to a guest post from Michigan theoretical computer scientist Seth Pettie, who writes about a SOSA Best Paper Award newly renamed in honor of the late Mihai Pătrașcu. Mihai, who I knew from his student days, was a brash, larger-than-life figure in theoretical computer science, for a brief few years until brain cancer tragically claimed him at the age of 29. Mihai and I didn’t always agree—indeed, I don’t think he especially liked me, or this blog—but as I wrote when he passed, his death made any squabbles seem trivial in retrospect. He was a lion of data structures, and it’s altogether fitting that this award be named for him. –SA

Seth’s guest post:

The SIAM Symposium on Simplicity in Algorithms (SOSA) was created in 2018 and has been awarding a Best Paper Award since 2020. This year the Steering Committee renamed this award after Mihai Pătrașcu, an extraordinary researcher in theoretical computer science who passed away before his time, in 2012.

Mihai’s research career lasted just a short while, from 2004-2012, but in that span of time he had a huge influence on research in geometry, graph algorithms, data structures, and especially lower bounds. He revitalized the entire areas of cell-probe lower bounds and succinct data structures, and laid the foundation for fine-grained complexity with the first 3SUM-hardness proof for graph problems. He lodged the most successful attack to date on the notorious dynamic optimality conjecture, then recast it
as a pure geometry problem. If you are too young to have met Mihai personally, I encourage you to pick up one of his now-classic papers. They are a real joy to read—playful and full of love for theoretical computer science.

The premise of SOSA is that simplicity is extremely valuable, rare, and inexplicably undervalued. We wanted to create a venue where the chief metrics of success were simplicity and insight. It is fitting that the SOSA Best Paper Award be named after Mihai. He brought “fresh eyes” to every problem he worked on, and showed that the cure for our problems is usually one key insight (and of course some mathematical gymnastics).

Let me end by thanking the SOSA 2026 Program Committee, co-chaired by Sepehr Assadi and Eva Rotenberg, and congratulating the authors of the SOSA 2026 Mihai Pătrașcu Best Paper:

• A Quasi-Polynomial Time Algorithm for 3-Coloring Circle Graphs
  Ajaykrishnan E S, Robert Ganian, Daniel Lokshtanov, Vaishali Surianarayanan

This award will be given at the SODA/SOSA business meeting in Vancouver, Canada, on January 12, 2026.

Yesterday I had a great conversation with Dmitri Tymoczko about groupoids in music theory. But at this Higgs Centre Colloquium, he preferred to downplay groupoids and talk in a way physicists would enjoy more. Click here to watch his talk!

What’s great is that Tymoczkyo not faking it: he’s really found deep ways in which symmetry shows up pervasively in music.

At first he tried to describe them geometrically using orbifolds, which are spaces in which some singular points have nontrivial symmetry groups, like the tip of a cone formed by modding out the plane by the action of the group $\mathbb{Z}/n$. But then he realized that the geometry was less important than the symmetry, which you can describe using groupoids. That’s why his talk is called “Beyond the geometry of music”.

I’m helping him with his work on groupoids, and I hope he explains his work to mathematicians someday without pulling his punches. I didn’t get to interview him yesterday, but I’ll try to do that soon.

For now you can read his books A Geometry of Music and Harmony: an Owner’s Manual along with many papers. What I’ve read so far is really exciting.

I had a piece in Quanta Magazine recently, about a tricky paradox that’s puzzling quantum gravity researchers and some early hints at its resolution.

The paradox comes from trying to describe “closed universes”, which are universes where it is impossible to reach the edge, even if you had infinite time to do it. This could be because the universe wraps around like a globe, or because the universe is expanding so fast no traveler could ever reach an edge. Recently, theoretical physicists have been trying to describe these closed universes, and have noticed a weird issue: each such universe appears to have only one possible quantum state. In general, quantum systems have more possible states the more complex they are, so for a whole universe to have only one possible state is a very strange thing, implying a bizarrely simple universe. Most worryingly, our universe may well be closed. Does that mean that secretly, the real world has only one possible state?

There is a possible solution that a few groups are playing around with. The argument that a closed universe has only one state depends on the fact that nothing inside a closed universe can reach the edge. But if nothing can reach the edge, then trying to observe the universe as a whole from outside would tell you nothing of use. Instead, any reasonable measurement would have to come from inside the universe. Such a measurement introduces a new kind of “edge of the universe”, this time not in the far distance, but close by: the edge between an observer and the rest of the world. And when you add that edge to the calculations, the universe stops being closed, and has all the many states it ought to.

This was an unusually tricky story for me to understand. I narrowly avoided several misconceptions, and I’m still not sure I managed to dodge all of them. Likewise, it was unusually tricky for the editors to understand, and I suspect it was especially tricky for Quanta’s social media team to understand.

It was also, quite clearly, tricky for the readers to understand. So I thought I would use this post to clear up a few misconceptions. I’ll say a bit more about what I learned investigating this piece, and try to clarify what the result does and does not mean.

Q: I’m confused about the math terms you’re using. Doesn’t a closed set contain its boundary?

A: Annoyingly, what physicists mean by a closed universe is a bit different from what mathematicians mean by a closed manifold, which is in turn more restrictive than what mathematicians mean by a closed set. One way to think about this that helped me is that in an open set you can take a limit that takes you out of the set, which is like being able to describe a (possibly infinite) path that takes you “out of the universe”. A closed set doesn’t have that, every path, no matter how long, still ends up in the same universe.

Q: So a bunch of string theorists did a calculation and got a result that doesn’t make sense, a one-state universe. What if they’re just wrong?

A: Two things:

First, the people I talked to emphasized that it’s pretty hard to wiggle out of the conclusion. It’s not just a matter of saying you don’t believe in string theory and that’s that. The argument is based in pretty fundamental principles, and it’s not easy to propose a way out that doesn’t mess up something even more important.

That’s not to say it’s impossible. One of the people I interviewed, Henry Maxfield, thinks that some of the recent arguments are misunderstanding how to use one of their core techniques, in a way that accidentally presupposes the one-state universe.

But even he thinks that the bigger point, that closed universes have only one state, is probably true.

And that’s largely due to a second reason: there are older arguments that back the conclusion up.

One of the oldest dates back to John Wheeler, a physicist famous for both deep musings about the nature of space and time and coining evocative terms like “wormhole”. In the 1960’s, Wheeler argued that, in a theory where space and time can be curved, one should think of a system’s state as including every configuration it can evolve into over time, since it can be tricky to specify a moment “right now”. In a closed universe, you could expect a quantum system to explore every possible configuration…meaning that such a universe should be described by only one state.

Later, physicists studying holography ran into a similar conclusion. They kept noticing systems in quantum gravity where you can describe everything that happens inside by what happens on the edges. If there are no edges, that seems to suggest that in some sense there is nothing inside. Apparently, Lenny Susskind had a slide at the end of talks in the 90’s where he kept bringing up this point.

So even if the modern arguments are wrong, and even if string theory is wrong…it still looks like the overall conclusion is right.

Q: If a closed universe has only one state, does that make it deterministic, and thus classical?

A: Oh boy…

So, on the one hand, there is an idea, which I think also goes back to Wheeler, that asks: “if the universe as a whole has a wavefunction, how does it collapse?” One possibility is that the universe has only one state, so that nobody is needed to collapse the wavefunction, it already is in a definite state.

On the other hand, a universe with only one state does not actually look much like a classical universe. Our universe looks classical largely due to a process called decoherence, where small quantum systems interact with big quantum systems with many states, diluting quantum effects until the world looks classical. If there is only one state, there are no big systems to interact with, and the world has large quantum fluctuations that make it look very different from a classical universe.

Q: How, exactly, are you defining “observer”?

A: A few commenters helpfully chimed in to talk about how physics models observers as “witness” systems, objects that preserve some record of what happens to them. A simple example is a ball sitting next to a bowl: if you find the ball in the bowl later, it means something moved it. This process, preserving what happens and making it more obvious, is in essence how physicists think about observers.

However, this isn’t the whole story in this case. Here, different research groups introducing observers are doing it in different ways. That’s, in part, why none of them are confident they have the right answer.

One of the approaches describes an observer in terms of its path through space and time, its worldline. Instead of a detailed witness system with specific properties, all they do is pick out a line and say “the observer is there”. Identifying that line, and declaring it different from its surroundings, seems to be enough to recover the complexity the universe ought to have.

The other approach treats the witness system in a bit more detail. We usually treat an observer in quantum mechanics as infinitely large compared to the quantum systems they measure. This approach instead gives the observer a finite size, and uses that to estimate how far their experience will be from classical physics.

Crucially, both approaches aren’t a matter of defining a physical object, and looking for it in the theory. Given a collection of atoms, neither team can tell you what is an observer, and what isn’t. Instead, in each approach, the observer is arbitrary: a choice, made by us when we use quantum mechanics, of what to count as an observer and what to count as the rest of the world. That choice can be made in many different ways, and each approach tries to describe what happens when you change that choice.

This is part of what makes this approach uncomfortable to some more philosophically-minded physicists: it treats observers not as a predictable part of the physical world, but as a mathematical description used to make statements about the world.

Q: If these ideas come from AdS/CFT, which is an open universe, how do you use them to describe a closed universe?

A: While more examples emerged later, initially theorists were thinking about two types of closed universes:

First, think about a black hole. You may have heard that when you fall into a black hole, you watch the whole universe age away before your eyes, due to the dramatic differences in the passage of time caused by the extreme gravity. Once you’ve seen the outside universe fade away, you are essentially in a closed universe of your own. The outside world will never affect you again, and you are isolated, with no path to the outside. These black hole interiors are one of the examples theorists looked at.

The other example are so-called “baby universes”. When physicists use quantum mechanics to calculate the chance of something happening, they have to add up every possible series of events that could have happened in between. For quantum gravity, this includes every possible arrangement of space and time. This includes arrangements with different shapes, including ones with tiny extra “baby universes” which branch off from the main universe and return. Universes with these “baby universes” are another example that theorists considered to understand closed universes.

Q: So wait, are you actually saying the universe needs to be observed to exist? That’s ridiculous, didn’t the universe exist long before humans existed to observe it? Is this some sort of Copenhagen Interpretation thing, or that thing called QBism?

You’re starting to ask philosophical questions, and here’s the thing:

There are physicists who spend their time thinking about how to interpret quantum mechanics. They talk to philosophers, and try to figure out how to answer these kinds of questions in a consistent and systematic way, keeping track of all the potential pitfalls and implications. They’re part of a subfield called “quantum foundations”.

The physicists whose work I was talking about in that piece are not those people.

Of the people I interviewed, one of them, Rob Myers, probably has lunch with quantum foundations researchers on occasion. The others, based at places like MIT and the IAS, probably don’t even do that.

Instead, these are people trying to solve a technical problem, people whose first inclination is to put philosophy to the side, and “shut up and calculate”. These people did a calculation that ought to have worked, checking how many quantum states they could find in a closed universe, and found a weird and annoying answer: just one. Trying to solve the problem, they’ve done technical calculation work, introducing a path through the universe, or a boundary around an observer, and seeing what happens. While some of them may have their own philosophical leanings, they’re not writing works of philosophy. Their papers don’t talk through the philosophical implications of their ideas in all that much detail, and they may well have different thoughts as to what those implications are.

So while I suspect I know the answers they would give to some of these questions, I’m not sure.

Instead, how about I tell you what I think?

I’m not a philosopher, I can’t promise my views will be consistent, that they won’t suffer from some pitfall. But unlike other people’s views, I can tell you what my own views are.

To start off: yes, the universe existed before humans. No, there is nothing special about our minds, we don’t have psychic powers to create the universe with our thoughts or anything dumb like that.

What I think is that, if we want to describe the world, we ought to take lessons from science.

Science works. It works for many reasons, but two important ones stand out.

Science works because it leads to technology, and it leads to technology because it guides actions. It lets us ask, if I do this, what will happen? What will I experience?

And science works because it lets people reach agreement. It lets people reach agreement because it lets us ask, if I observe this, what do I expect you to observe? And if we agree, we can agree on the science.

Ultimately, if we want to describe the world with the virtues of science, our descriptions need to obey this rule: they need to let us ask “what if?” questions about observations.

That means that science cannot avoid an observer. It can often hide the observer, place them far away and give them an infinite mind to behold what they see, so that one observer is essentially the same as another. But we shouldn’t expect to always be able to do this. Sometimes, we can’t avoid saying something about the observer: about where they are, or how big they are, for example.

These observers, though, don’t have to actually exist. We should be able to ask “what if” questions about others, and that means we should be able to dream up fictional observers, and ask, if they existed, what would they see? We can imagine observers swimming in the quark-gluon plasma after the Big Bang, or sitting inside a black hole’s event horizon, or outside our visible universe. The existence of the observer isn’t a physical requirement, but a methodological one: a restriction on how we can make useful, scientific statements about the world. Our theory doesn’t have to explain where observers “come from”, and can’t and shouldn’t do that. The observers aren’t part of the physical world being described, they’re a precondition for us to describe that world.

Is this the Copenhagen Interpretation? I’m not a historian, but I don’t think so. The impression I get is that there was no real Copenhagen Interpretation, that Bohr and Heisenberg, while more deeply interested in philosophy than many physicists today, didn’t actually think things through in enough depth to have a perspective you can name and argue with.

Is this QBism? I don’t think so. It aligns with some things QBists say, but they say a lot of silly things as well. It’s probably some kind of instrumentalism, for what that’s worth.

Is it logical positivism? I’ve been told logical positivists would argue that the world outside the visible universe does not exist. If that’s true, I’m not a logical positivist.

Is it pragmatism? Maybe? What I’ve seen of pragmatism definitely appeals to me, but I’ve seen my share of negative characterizations as well.

In the end, it’s an idea about what’s useful and what’s not, about what moves science forward and what doesn’t. It tries to avoid being preoccupied with unanswerable questions, and as much as possible to cash things out in testable statements. If I do this, what happens? What if I did that instead?

The results I covered for Quanta, to me, show that the observer matters on a deep level. That isn’t a physical statement, it isn’t a mystical statement. It’s a methodological statement: if we want to be scientists, we can’t give up on the observer.

read more

Should you require a model for an Oxford don in a play or novel, look no farther than Andrew Briggs. The emeritus professor of nanomaterials speaks with a southern-English accent as crisp as shortbread, exhibits manners to which etiquette influencer William Hanson could aspire, and can discourse about anything from Bantu to biblical Hebrew. I joined Andrew for lunch at St. Anne’s College, Oxford, this month.¹ Over vegetable frittata, he asked me what unifying principle distinguishes quantum from classical thermodynamics.

I’d approached quantum thermodynamics from nearly every angle I could think of. I’d marched through the thickets of derivations and plots; I’d journeyed from subfield to subfield; I’d gazed down upon the discipline as upon a landscape from a hot-air balloon. I’d even prepared a list of thermodynamic tasks enhanced by quantum phenomena: we can charge certain batteries at greater powers if we entangle them than if we don’t, entanglement can raise the amount of heat pumped out of a system by a refrigerator, etc. But Andrew’s question flummoxed me.

I bungled the answer. I toted out the aforementioned list, but it contained examples, not a unifying principle. The next day, I was sitting in an office borrowed from experimentalist Natalia Ares in New College, a Gothic confection founded during the late 1300s (as one should expect of a British college called “New”). Admiring the view of ancient stone walls, I realized how I should have responded the previous day.

My answer begins with a blog post written in response to a quantum-thermodynamics question from a don at another venerable university: Yoram Alhassid. He asked, “What distinguishes quantum thermodynamics to quantum statistical mechanics?” You can read the full response here. Takeaways include thermodynamics’s operational flavor. When using an operational theory, we imagine agents who perform tasks, using given resources. For example, a thermodynamic agent may power a steamboat, given a hot gas and a cold gas. We calculate how effectively the agents can perform those tasks. For example, we compute heat engines’ efficiencies. If a thermodynamic agent can access quantum resources, I’ll call them “quantum thermodynamic.” If the agent can access only everyday resources, I’ll call them “classical thermodynamic.”

A quantum thermodynamic agent may access more resources than a classical thermodynamic agent can. The latter can leverage work (well-organized energy), free energy (the capacity to perform work), information, and more. A quantum agent may access not only those resources, but also entanglement (strong correlations between quantum particles), coherence (wavelike properties of quantum systems), squeezing (the ability to toy with quantum uncertainty as quantified by Heisenberg and others), and more. The quantum-thermodynamic agent may apply these resources as described in the list I rattled off at Andrew.

Yet quantum phenomena can impede a quantum agent in certain scenarios, despite assisting the agent in others. For example, coherence can reduce a quantum engine’s power. So can noncommutation. Everyday numbers commute under multiplication: 11 times 12 equals 12 times 11. Yet quantum physics features numbers that don’t commute so. This noncommutation underlies quantum uncertainty, quantum error correction, and much quantum thermodynamics blogged about ad nauseam on Quantum Frontiers. A quantum engine’s dynamics may involve noncommutation (technically, the Hamiltonian may contain terms that fail to commute with each other). This noncommutation—a fairly quantum phenomenon—can impede the engine similarly to friction. Furthermore, some quantum thermodynamic agents must fight decoherence, the leaking of quantum information from a quantum system into its environment. Decoherence needn’t worry any classical thermodynamic agent.

In short, quantum thermodynamic agents can benefit from more resources than classical thermodynamic agents can, but the quantum agents also face more threats. This principle might not encapsulate how all of quantum thermodynamics differs from its classical counterpart, but I think the principle summarizes much of the distinction. And at least I can posit such a principle. I didn’t have enough experience when I first authored a blog post about Oxford, in 2013. People say that Oxford never changes, but this quantum thermodynamic agent does.

In the University of Oxford Natural History Museum in 2013, 2017, and 2025. I’ve published nearly 150 Quantum Frontiers posts since taking the first photo!

¹Oxford consists of colleges similarly to how neighborhoods form a suburb. Residents of multiple neighborhoods may work in the same dental office. Analogously, faculty from multiple colleges may work, and undergraduates from multiple colleges may major, in the same department.

 (Apologies for the ugly blog format. We had a bit of a crash, and are working to get the template back in working order.)

This year we give thanks for a crucially important idea that can mean very different things to different people: information. (We’ve previously given thanks for the Standard Model Lagrangian, Hubble’s Law, the Spin-Statistics Theorem, conservation of momentum, effective field theory, the error bar, gauge symmetry, Landauer’s Principle, the Fourier Transform, Riemannian Geometry, the speed of light, the Jarzynski equality, the moons of Jupiter, space, black hole entropy, electromagnetism, Arrow’s Impossibility Theorem, and quanta.)

“Information” is an idea that is everywhere in science and technology these days. From one angle it looks like such an obvious idea that it’s a bit startling to realize that information theory didn’t really come along until the work of Claude Shannon in the 1940s. From another, the idea has so many different shades of meaning that we shouldn’t be surprised (that’s a joke you will get in a bit) that it can be hard to understand.

Information theory is obviously an enormous subject, but we’re just giving thanks, not writing a textbook. I want to mention two ideas I find especially central. First, Shannon’s idea about relating information content to “surprisal.” Second, the very different intuitive notions of information that we get from engineering and physics.

Shannon, working at Bell Labs, was interested in the problem of how to send trustworthy signals efficiently over transatlantic cables. He was thinking about various ways to express information in a code: a set of symbols, each with a defined meaning. So a code might be an alphabet, or a set of words, or a literal cipher. And he noticed that there was a lot of redundancy in natural languages; the word “the” appears much more often in English than the word “axe,” although both have the same number of letters.

Let’s refer to each letter or symbol in a code as an “event.” Shannon’s insight was to realize that the more unlikely an event, the more information it conveyed when it was received. The statements “The Sun rose in the east this morning” and “The Sun rose in the west this morning” contain the same number of letters, but the former contains almost no information — you already were pretty sure the Sun would be rising in the east. But the latter, if obtained from a reliable source, would be very informative indeed, precisely because it was so unexpected. Clearly some kind of unprecedented astronomical catastrophe was in progress.

Imagine we can assign a probability to every different event . Shannon wanted a way to quantify the information content of that event, which would satisfy various reasonable-seeming axioms: most crucially, that the information content of two independent events is the sum of the individual information contents. But the joint probability of two events is the product of their individual probabilities. So the natural thing to do would be to define the information content as the logarithm of the probability; the logarithm of a product equals the sum of the individual logarithms. But you want low probability to correspond to high information content, so Shannon defined the information content (also called the self-information, or surprisal, or Shannon information) of an event to be minus the log of the probability, which by math is equal to the log of the reciprocal of the probability:

Note that probabilities are numbers between 0 and 1, and the log of such a number will be negative, with numbers closer to 0 being more negative than numbers closer to 1. So goes from at to at . An impossible message is infinitely surprising, and therefore conveys infinite information; an inevitable message is completely unsurprising, and conveys no information at all.

From there, Shannon suggested that we could characterize how efficient an entire code was at conveying information: just calculate the average (expectation value) of the information content for all possible events. When we have a probability distribution , the average of any function is just the sum of the the values of the function times their respective probabilities, . So we characterize the information content of a code via the quantity

The only question is, what to call this lovely newly-defined quantity that surely nobody had ever thought of before? Happily Shannon was friends with John von Neumann, who informed him, “You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage.” So entropy it is.

Indeed, this formula is precisely that which had been put forward (unknown to Shannon) by Josiah Willard Gibbs in the 1870’s as a definition of entropy in statistical mechanics. (It is related to the definition on Ludwig Boltzmann’s tombstone, , and Boltzmann had also suggested similar expressions to the above.) On the one hand, it seems remarkable to find precisely the same expression playing central roles in problems as disparate as sending signals across cables and watching cream mix into coffee; on the other hand, it’s a relatively simple expression and the axioms used to derive it are actually pretty similar, so perhaps we shouldn’t be surprised; on the third hand, the connection between information theory and statistical mechanics turns out to be deep and fruitful, so it’s more than just a mathematical coincidence.

But let me highlight the one aspect of the term “information” that can be sometimes confusing to people. To the engineer, a code that is maximally informative is one for which is relatively uniform over all events , which means is maximal or close to it; in that case, every event will tell you something at least a little bit interesting. For them, high entropy = high information.

But to a physicist who might be asking “how much information do I have about the state of a system?”, you have more information when is relatively narrowly concentrated around some value, rather than being all spread out. For them, high entropy = low information! Indeed, one physically-relevant notion of “information” is the “accessible information” of a system, which can be defined as . (I talk about this a bit in my recent solo podcast on complexity.)

Perhaps we shouldn’t be so surprised that physicists and engineers posit oppositely-directed relationships between entropy and information. It’s just a reflection of the fact that “information” is so ubiquitous and has so many different uses. We should be thankful that we’re beginning to understand it so well.

It’s been over three years since my last post on this blog and I have sometimes been asked, understandably, whether the project I announced in my previous post was actually happening. The answer is yes — the grant I received from the Astera Institute has funded several PhD students and a couple of postdocs, and we have been busy. In my previous post I suggested that I would be open to remote collaboration, but that has happened much less, partly because a Polymath-style approach would have been difficult to manage while also ensuring that my PhD students would have work that they could call their own to put in their theses.

In general I don’t see a satisfactory solution to that problem, but in this post I want to mention a subproject of the main project that is very much intended to be a large public collaboration. A few months ago, a call came out from Renaissance Philanthropies saying that they were launching a $9m AI for Math Fund to spend on projects in the general sphere of AI and mathematics, and inviting proposals. One of the categories that they specifically mentioned was creating new databases, and my group submitted a proposal to create a database of what we call “structured motivated proofs,” a piece of terminology that I will explain a bit more later in just a moment. I am happy to report that our proposal was one of the 29 successful ones. Since a good outcome to the project will depend on collaboration from many people outside the group, we need to publicize it, which is precisely the purpose of this post. Below I will be more specific about the kind of help we are looking for.

I spent the day at the NSBP / NSHP meeting in San José. My favorite session of the day was the morning astro session, which was entirely about brown dwarfs. I learned a lot in a very short time. Caprice Phillips (UCSC) introduced the session with an introduction to the scientific and technical questions in play. She put a lot of emphasis on using binaries and clusters to put detailed abundance ratios onto substellar objects. This was what I expected: I thought (walking in to this session) that all known abundance ratios for brown dwarfs were from such kinds of studies. I learned different (keep reading).

Gabriel Munoz Zarazua (SFSU) followed by showing spectra from M-dwarfs, brown dwarfs, and Jupiter. It definitely looks like a sequence. He does spectral fitting (what they call, in this business, retrievals). It looks like he is getting very good, somewhat precise, abundance ratios for the photospheres of substellar objects! I asked more about this in the question period, and apparently I am way behind the times (Emily Rauscher, Michigan, helpfully pointed this out to me): Now brown-dwarf photosphere models are so good, they can be used to measure abundances, and pretty well.

I also learned in this session (maybe from Jorge Sanchez, ASU, or maybe from Efrain Alvarado, SFSU) that there is a very strong mass–abundance relation in the Solar System. That is, we don't expect, if brown dwarfs form the way planets do, that the detailed abundances of the brown dwarfs will match exactly the detailed abundances of the primary stars. But now we are really in a position to test that. Sanchez showed that we can get, from even photometry, abundances for substellar objects in the Milky Way halo. Again, totally new to me! And he finds metallicities at or below −3. Alvarado showed data on an amazing system J1416, which is an L–T binary with no stellar companion. Apparently it is the only known completely substellar binary.

read more

A 9-year-old named Kai (“The Quantum Kid”) and his mother interviewed me about closed timelike curves, wormholes, Deutsch’s resolution of the Grandfather Paradox, and the implications of time travel for computational complexity:

This is actually one of my better podcasts (and only 24 minutes long), so check it out!

Here’s a podcast I did a few months ago with “632nm” about P versus NP and my other usual topics:

For those who still can’t get enough, here’s an interview about AI alignment for the “Hidden Layers” podcast that I did a year ago, and that I think I forgot to share on this blog at the time:

What else is in the back-catalog? Ah yes: the BBC interviewed me about quantum computing for a segment on Moore’s Law.

As you may have heard, Steven Pinker recently wrote a fantastic popular book about the concept of common knowledge, entitled When Everyone Knows That Everyone Knows… Steve’s efforts render largely obsolete my 2015 blog post Common Knowledge and Aumann’s Agreement Theorem, one of the most popular posts in this blog’s history. But I’m willing to live with that, not only because Steven Pinker is Steven Pinker, but also because he used my post as a central source for the topic. Indeed, you should watch his podcast with Richard Hanania, where Steve lucidly explains Aumann’s Agreement Theorem, noting how he first learned about it from this blog.

Yesterday I had a great conversation with Dmitri Tymoczko about groupoids in music theory. But at this Higgs Centre Colloquium, he preferred to downplay groupoids and talk in a way physicists would enjoy more. Click on the picture to watch his talk!

What’s great is that he’s not faking it: he’s really found deep ways in which symmetry shows up pervasively in music.

At first he tried to describe them geometrically using ‘orbifolds’, which are spaces in which some singular points have nontrivial symmetry groups, like the tip of a cone. But then he realized that the geometry was less important than the symmetry, which you can describe using ‘groupoids’: categories where every morphism is invertible. That’s why his talk is called “Beyond the geometry of music”.

I’m helping him with his work on groupoids, and I hope he explains his work to mathematicians someday without pulling his punches. I didn’t get to interview him yesterday, but I’ll try to do that soon.

For now you can read his books A Geometry of Music and Harmony: an Owner’s Manual along with many papers. What I’ve read so far is really exciting.

Sometimes, the world is silly for honest, happy reasons. And sometimes, it’s silly for reasons you never even considered.

Scientific projects often have acronyms, some of which are…clever, let’s say. Astronomers are famous for acronyms. Read this list, and you can find examples from 2D-FRUTTI and ABRACADABRA to WOMBAT and YORIC. Some of these aren’t even “really” acronyms, using letters other than the beginning of each word, multiple letters from a word, or both. (An egregious example from that list: VESTALE from “unVEil the darknesS of The gAlactic buLgE”.)

But here’s a pattern you’ve probably not noticed. I suggest that you should see more of these…clever…acronyms in projects in Europe, and they should show up in a wider range of fields, not just astronomy. And the reason why, is the European Research Council.

In the US, scientific grants are spread out among different government agencies. Typical grants are small, the kind of thing that lets a group share a postdoc every few years, with different types of grants covering projects of different scales.

The EU, instead, has the European Research Council, or ERC, with a flagship series of grants covering different career stages: Starting, Consolidator, and Advanced. Unlike most US grants, these are large (supporting multiple employees over several years), individual (awarded to a single principal investigator, not a collaboration) and general (the ERC uses the same framework across multiple fields, from physics to medicine to history).

That means there are a lot of medium-sized research projects in Europe that are funded by an ERC grant. And each of them are required to have an acronym.

Why? Who knows? “Acronym” is simply one of the un-skippable entries in the application forms, with a pre-set place of honor in their required grant proposal format. Nobody checks whether it’s a “real acronym”, so in practice it often isn’t, turning into some sort of catchy short name with “acronym vibes”. It, like everything else on these forms, is optimized to catch the attention of a committee of scientists who really would rather be doing something else, often discussed and refined by applicants’ mentors and sometimes even dedicated university staff.

So if you run into a scientist in Europe who proudly leads a group with a cutesy, vaguely acronym-adjacent name? And you keep running into these people?

It’s not a coincidence, and it’s not just scientists’ sense of humor. It’s the ERC.

Near the end of my last post, I made a little offhand remark:

[G]iven the current staggering rate of hardware progress, I now think it’s a live possibility that we’ll have a fault-tolerant quantum computer running Shor’s algorithm before the next US presidential election. And I say that not only because of the possibility of the next US presidential election getting cancelled, or preempted by runaway superintelligence!

As I later clarified, I’ll consider this “live possibility” to be fulfilled even if a fault-tolerant Shor’s algorithm is “merely” used to factor 15 into 3×5—a milestone that seems a few steps, but only a few steps, away from what Google, Quantinuum, QuEra, and others have already demonstrated over the past year. After that milestone, I then expect “smooth sailing” to more and more logical qubits and gates and the factorization of larger and larger integers, however fast or slow that ramp-up proceeds (which of course I don’t know).

In any case, the main reason I made my remark was just to tee up the wisecrack about whether I’m not sure if there’ll be a 2028 US presidential election.

My remark, alas, then went viral on Twitter, with people posting countless takes like this:

A quantum expert skeptic who the bears quote all the time – Scott Aaronson – recently got very excited about a number of quantum advances. He now thinks there’s a possibility of running Shor before the next US president election – a timeline that lines up ONLY with $IONQ‘s roadmap, and NOBODY else’s! This represent a MAJOR capitulation of previously predicted timelines by any skeptics.

Shall we enumerate the layers of ugh here?

1. I’ve been saying for several years now that anyone paranoid about cybersecurity should probably already be looking to migrate to quantum-resistant cryptography, because one can’t rule out the possibility that hardware progress will be fast. I didn’t “capitulate”: I mildly updated what I said before, in light of exciting recent advances.
   
2. A “live possibility” is short not only of a “certainty,” but of a “probability.” It’s basically just an “I’m not confident this won’t happen.”
   
3. Worst is the obsessive focus on IonQ, a company that I never mentioned (except in the context of its recently-acquired subsidiary, Oxford Ionics), but which now has a $17 billion valuation. I should explain that, at least since it decided to do an IPO, IonQ has generally been regarded within the research community as … err … a bit like the early D-Wave, intellectual-respectability-wise. They’ll eagerly sell retail investors on the use of quantum computers to recognize handwriting and suchlike, despite (I would say) virtually no basis to believe in a quantum scaling advantage for such tasks. Or they’ll aggressively market current devices to governments who don’t understand what they’re for, but just want to say they have a quantum computer and not get left behind. Or they’ll testify to Congress that quantum, unlike AI, “doesn’t hallucinate” and indeed is “deterministic.” It pains me to write this, as IonQ was founded by (and indeed, still employs) scientists who I deeply admire and respect.
   
4. Perhaps none of this would matter (or would matter only to pointy-headed theorists like me) if IonQ were the world leader in quantum computing hardware, or even trapped-ion hardware. But by all accounts, IonQ’s hardware and demonstrations have lagged well behind those of its direct competitor, Quantinuum. It seems to me that, to whatever extent IonQ gets vastly more attention, it’s mostly just because it chose to IPO early, and also because it’s prioritized marketing to the degree it has.

Over the past few days, I’ve explained the above to various people, only to have them look back at me with glazed, uncomprehending eyes and say, “so then, which quantum stock should I buy? or should I short quantum?”

It would seem rude for me to press quarters into these people’s hands, explaining that they must make gain from whatever they learn. So instead I reply: “You do realize, don’t you, that I’m, like, a professor at a state university, who flies coach and lives in a nice but unremarkable house? If I had any skill at timing the market, picking winners, etc., don’t you think I’d live in a mansion with an infinity pool, and fly my Cessna to whichever conferences I deigned to attend?”

It’s like this: if you think quantum computers able to break 2048-bit cryptography within 3-5 years are a near-certainty, then I’d say your confidence is unwarranted. If you think such quantum computers, once built, will also quickly revolutionize optimization and machine learning and finance and countless other domains beyond quantum simulation and cryptanalysis—then I’d say that more likely than not, an unscrupulous person has lied to you about our current understanding of quantum algorithms.

On the other hand, if you think Bitcoin, and SSL, and all the other protocols based on Shor-breakable cryptography, are almost certainly safe for the next 5 years … then I submit that your confidence is also unwarranted. Your confidence might then be like most physicists’ confidence in 1938 that nuclear weapons were decades away, or like my own confidence in 2015 that an AI able to pass a reasonable Turing Test was decades away. It might merely be the confidence that “this still looks like the work of decades—unless someone were to gather together all the scientific building blocks that have now been demonstrated, and scale them up like a stark raving madman.” The trouble is that sometimes people, y’know, do that.

Beyond that, the question of “how many years?” doesn’t even interest me very much, except insofar as I can mine from it the things I value in life, like scientific understanding, humor, and irony.

There are, famously, many intellectual Communists who are ruthless capitalists in their day-to-day lives. I somehow wound up the opposite. Intellectually, I see capitalism as a golden goose, a miraculous engine that’s lifted the human species out of its disease-ridden hovels and into air-conditioned high-rises, whereas Communism led instead to misery and gulags and piles of skulls every single time it was tried.

And yet, when I actually see the workings of capitalism up close, I often want to retch. In case after case, it seems, our system rewards bold, confident, risk-taking ignoramuses and liars, those who can shamelessly hype a technology (or conversely, declare it flatly impossible)—with such voices drowning out the cautious experts who not only strive to tell the truth, but also made all the actual discoveries that the technology rests on. My ideal economic system is, basically, whichever one can keep the people who can clearly explain the capabilities and limits and risks and benefits of X in charge of X for as long as possible.

60 people, including a lot of category theorists, are meeting in Edinburgh for the £59 million UK project called Safeguarded AI. I talked about it before here.

The plan is to build software that will let you precisely specify systems of many kinds, which an AI will design, and verify that what the AI designed meets your specifications. So: it’s not about building an AI, but instead, building a way to specify jobs for it and verify that it did those jobs correctly!

The director of this project, David Dalrymple, has changed the plan recently. There were many teams of category theorists designing formalisms to get this job done. David Jaz Myers at Topos Research UK was supposed to integrate all these formalisms. That would be a huge job.

But recently all but a few teams have been cut off from the main project—they can now do whatever they want. The project will focus on 3 parts:

1) The “categorical core”: a software infrastructure that lets you program using category theory concepts. I think Amar Hadzihasanovic, my former student Owen Lynch, and two others will be building this.

2) “DOTS”: the double operadic theory of systems, a general framework for building systems out of smaller parts. This is David Jaz Myers’ baby—see the videos.