Skip to the Main Content

Note:These pages make extensive use of the latest XHTML and CSS Standards. They ought to look great in any standards-compliant modern browser. Unfortunately, they will probably look horrible in older browsers, like Netscape 4.x and IE 4.x. Moreover, many posts use MathML, which is, currently only supported in Mozilla. My best suggestion (and you will thank me when surfing an ever-increasing number of sites on the web which have been crafted to use the new standards) is to upgrade to the latest version of your browser. If that's not possible, consider moving to the Standards-compliant and open-source Mozilla browser.

November 23, 2021

Compositional Thermostatics

Posted by John Baez

At the Topos Institute this summer, a group of folks started talking about thermodynamics and category theory. It probably started because Spencer Breiner and my former student Joe Moeller, both working at the National Institute of Standards and Technology, were talking about thermodynamics with some people there. But I’ve been interested in thermodynamics for quite a while now — and Owen Lynch, a grad student visiting from the University of Utrecht, wanted to do his master’s thesis on the subject. He’s now working with me. Sophie Libkind, David Spivak and David Jaz Myers also joined in: they’re especially interested in open systems and how they interact.

Prompted by these conversations, a subset of us eventually wrote a paper on the foundations of equilibrium thermodynamics:

Our goal was to describe classical thermodynamics, classical statistical mechanics and quantum statistical mechanics in a unified framework based on entropy maximization. This framework can also handle ‘generalized probabilistic theories’ of the sort studied in quantum foundations—that is, theories like quantum mechanics, but more general.

To unify all these theories, we define a ‘thermostatic system’ to be any convex space XX of ‘states’ together with a concave function

S:X[,] S \colon X \to [-\infty, \infty]

assigning to each state an ‘entropy’.

Whenever several such systems are combined and allowed to come to equilibrium, the new equilibrium state maximizes the total entropy subject to constraints. We explain how to express this idea using an operad. Intuitively speaking, the operad we construct has as operations all possible ways of combining thermostatic systems. For example, there is an operation that combines two gases in such a way that they can exchange energy and volume, but not particles — and another operation that lets them exchange only particles, and so on.

It is crucial to use a sufficiently general concept of ‘convex space’, which need not be a convex subset of a vector space. Luckily there has been a lot of work on this, so we can just grab a good definition off the shelf:

Definition. A convex space is a set XX with an operation c λ:X×XXc_\lambda \colon X \times X \to X for each λ[0,1]\lambda \in [0, 1] such that the following identities hold:

1) c 1(x,y)=xc_1(x, y) = x

2) c λ(x,x)=xc_\lambda(x, x) = x

3) c λ(x,y)=c 1λ(y,x)c_\lambda(x, y) = c_{1-\lambda}(y, x)

4) c λ(c μ(x,y),z)=c λ(x,c μ(y,z))c_\lambda(c_\mu(x, y) , z) = c_{\lambda'}(x, c_{\mu'}(y, z)) for all 0λ,μ,λ,μ10 \le \lambda, \mu, \lambda', \mu' \le 1 satisfying λμ=λ\lambda\mu = \lambda' and 1λ=(1λ)(1μ)1-\lambda = (1-\lambda')(1-\mu').

To understand these axioms, especially the last, you need to check that any vector space is a convex space with

c λ(x,y)=λx+(1λ)yc_\lambda(x, y) = \lambda x + (1-\lambda)y

So, these operations c λc_\lambda describe ‘convex linear combinations’.

Indeed, any subset of a vector space closed under convex linear combinations is a convex space! But there are other examples too.

In 1949, the famous mathematician Marshall Stone invented ‘barycentric algebras’. These are convex spaces satisfying one extra axiom: the cancellation axiom, which says that whenever λ0,\lambda \ne 0,

c λ(x,y)=c λ(x,y)x=xc_\lambda(x,y) = c_\lambda(x',y) \implies x = x'

He proved that any barycentric algebra is isomorphic to a convex subset of a vector space. Later Walter Neumann noted that a convex space, defined as above, is isomorphic to a convex subset of a vector space if and only if the cancellation axiom holds.

Dropping the cancellation axiom has convenient formal consequences, since the resulting more general convex spaces can then be defined as algebras of a finitary commutative monad, giving the category of convex spaces very good properties.

But dropping this axiom is no mere formal nicety. In our definition of ‘thermostatic system’, we need the set of possible values of entropy to be a convex space. One obvious candidate is the set [0,).[0,\infty). However, for a well-behaved formalism based on entropy maximization, we want the supremum of any set of entropies to be well-defined. This forces us to consider the larger set [0,],[0,\infty], which does not obey the cancellation axiom.

But even that is not good enough! In thermodynamics you often read about the ‘heat bath’, an idealized system that can absorb or emit an arbitrarily large amount of energy while keeping a fixed temperature. We want to treat the ‘heat bath’ as a thermostatic system on an equal footing with any other. To do this, we need to allow consider negative entropies—not because the heat bath can have negative entropy, but because it acts as an infinite reservoir of entropy, and the change in entropy from its default state can be positive or negative.

This suggests letting entropies take values in the convex space .\mathbb{R}. But then the requirement that any set of entropies have a supremum (including empty and unbounded sets) forces us to use the larger convex space [,].[-\infty,\infty].

This does not obey the cancellation axiom, so there is no way to think of it as a convex subset of a vector space. In fact, it’s not even immediately obvious how to make it into a convex space at all! After all, what do you get when you take a nontrivial convex linear combination of \infty and ?-\infty? You’ll have to read our paper for the answer, and the justification.

We then define a thermostatic system to be a convex set XX together with a concave function

S:X[,] S \colon X \to [-\infty, \infty]

where concave means that

S(c λ(x,y))c λ(S(x),S(y)) S(c_\lambda(x,y)) \ge c_\lambda(S(x), S(y))

We give lots of examples from classical thermodynamics, classical and quantum statistical mechanics, and beyond—including our friend the ‘heat bath’.

For example, suppose XX is the set of probability distributions on an nn-element set, and suppose S:X[,]S \colon X \to [-\infty, \infty] is the Shannon entropy

S(p)= i=1 np ilogp i \displaystyle{ S(p) = - \sum_{i = 1}^n p_i \log p_i }

Then given two probability distributions pp and q,q, we have

S(λp+(1λq))λS(p)+(1λ)S(q) S(\lambda p + (1-\lambda q)) \ge \lambda S(p) + (1-\lambda) S(q)

for all λ[0,1].\lambda \in [0,1]. So this entropy function is convex, and S:X[,]S \colon X \to [-\infty, \infty] defines a thermostatic system. But in this example the entropy only takes nonnegative values, and is never infinite, so you need to look at other examples to see why we want to let entropy take values in [,].[-\infty,\infty].

After looking at examples of thermostatic systems, we define an operad whose operations are convex-linear relations from a product of convex spaces to a single convex space. And then we prove that thermostatic systems give an algebra for this operad: that is, we can really stick together thermostatic systems in all these ways. The trick is computing the entropy function of the new composed system from the entropy functions of its parts: this is where entropy maximization comes in.

For a nice introduction to these ideas, check out Owen’s blog article:

And then comes the really interesting part: checking that this adequately captures many of the examples physicists have thought about!

Here’s one: two cylinders of ideal gas with a movable divider between them that’s permeable to heat.

Yes, this is an operation in our operad—and if you tell us the entropy function of each cylinder of gas, our formalism will automatically compute the entropy function of the resulting combination of these two cylinders.

There are many other examples. Did you ever hear of the ‘canonical ensemble’, the ‘microcanonical ensemble’, or the ‘grand canonical ensemble’? Those are famous things in statistical mechanics. We show how our formalism recovers these.

I’m sure there’s much more to be done. But I feel happy to see modern math being put to good use: making the foundations of thermodynamics more precise. Once Vladimir Arnol’d wrote:

Every mathematician knows that it is impossible to understand any elementary course in thermodynamics.

I’m not sure our work will help with that — and indeed, it’s possible that once the mathematicians finally understand thermodynamics, physicists won’t understand what the mathematicians are talking about! But at least we’re clearly seeing some more of the mathematical structures that are hinted at, but not fully spelled out, in such an elementary course.

I expect that our work will interact nicely with Simon’s work on the Legendre transform. The Legendre transform of a concave (or convex) function is widely used in thermostatics, and Simon describes this for functions valued in [,][-\infty,\infty] using enriched profunctors:

He also has a paper on this, and you can see him talk about it on YouTube.

Posted at November 23, 2021 6:44 PM UTC

TrackBack URL for this Entry:   https://golem.ph.utexas.edu/cgi-bin/MT-3.0/dxy-tb.fcgi/3367

16 Comments & 0 Trackbacks

Re: Compositional Thermostatics

Jumped the gun, sorry, posted before I got to the refs to Simon Willerton’s work; molodets!

Posted by: jackjohnson on November 23, 2021 9:33 PM | Permalink | Reply to this

Re: Compositional Thermostatics

After all, what do you get when you take a nontrivial convex linear combination of \infty and −\infty?

Since a system with −\infty entropy is just a heat bath, wouldn’t a convex linear combination of anything and −\infty be −\infty?

Posted by: Madeleine Birchfield on November 24, 2021 6:18 PM | Permalink | Reply to this

Re: Compositional Thermostatics

A system with entropy -\infty is not a heat bath, at least not in our definition of heat bath (Example 15), but you’re right that we define any nontrivial convex combination of \infty and -\infty to be -\infty, and this really matters.

For us a heat bath at some temperature T 00T_0 \ne 0 is the thermostatic system whose convex space of states is \mathbb{R}, where a point UU \in \mathbb{R} indicates the amount of energy added to the heat bath from the rest of the world, and whose entropy function is

S(U)=U/T 0 S(U) = U/T_0

where this means the extra entropy of the heat bath due to adding energy to it. So, the entropy of the heat bath can be any real number, depending on UU.

We make this definition of the heat bath to fit what physicists normally do with the concept of heat bath. We’re proud of having formalized a concept that’s often left a bit fuzzy.

Posted by: John Baez on November 25, 2021 12:08 AM | Permalink | Reply to this

Re: Compositional Thermostatics

Getzler and Kapranov, in \SS 7 - 8 of

Modular operads arXiv:dg-ga/9408003

(published later in Compositio Math.) define an interesting Legendre transform on the ring of classical symmetric functions.

Posted by: jackjohnson on November 24, 2021 9:43 PM | Permalink | Reply to this

Re: Compositional Thermostatics

Thanks! I’m studying Legendre transforms and symmetric functions in what I thought were two completely unrelated projects: this paper and the paper Schur functors and categorified plethysm. It’s scary to think that they could be related (though you’ve hinted at this before).

Posted by: John Baez on November 25, 2021 2:59 AM | Permalink | Reply to this

Re: Compositional Thermostatics

You’ve been telling us for a while that thermodynamics, statistical mechanics, and information theory share a lot of DNA; I’m just seconding the motion.


Perhaps a thought for today?: If you give a man a fish you feed him for the day, but if you suggest he put a little fish in with his corn kernel he will steal all your lands and take all your women.

Posted by: jackjohnson on November 25, 2021 5:58 PM | Permalink | Reply to this

Re: Compositional Thermostatics

But the connection to symmetric functions is mysterious and downright scary to me!

Posted by: John Baez on November 26, 2021 3:55 PM | Permalink | Reply to this

Re: Compositional Thermostatics

I can’t claim to understand either G-K’s or SW’s Legendre transform but it occurred to me to wonder (I will probably regret mentioning this) if the latter might be some kind of tropicalization of the former; perhaps a project for next year… There’s even yet another conceivably related theory of (noncommutative)

https://en.wikipedia.org/wiki/Free_probability

that I know less than nothing about.

Posted by: jackjohnson on November 27, 2021 1:21 AM | Permalink | Reply to this

Re: Compositional Thermostatics

Looking back through the mists of time, we were speaking of the Legendre transform as the tropicalization of the Laplace transform.

Posted by: David Corfield on November 27, 2021 3:54 PM | Permalink | Reply to this

Re: Compositional Thermostatics

Yes, I remember that, and it’s connected to my new work on thermodynamics versus classical mechanics… in ways I should explain!

Posted by: John Baez on November 28, 2021 1:59 AM | Permalink | Reply to this

Re: Compositional Thermostatics

This looks very interesting, of course.

Here’s a quick question. You take the category of convex spaces and convex relations. I might guess (!) that this could be a subcategory of the category where the objects are convex spaces and the morphisms from XX to YY are the concave functions from X×YX\times Y to ¯\overline{\mathbb{R}}, with composition given by a convolution

(gf)(x,z)=sup y{g(y,z)+f(x,y)}(g\circ f)(x, z)= \sup_y\{g(y, z) + f(x, y)\}

or something similar (I haven’t thought through the details but it would require the composite of concave functions is concave). The convex relations would sit inside this by taking a subset PX×YP\subset X\times Y to its indicator function: I P(x,y)=0I_P(x,y)= 0 if (x,y)P(x,y)\in P and -\infty otherwise.

We could call this category ConvPair for ‘convex pairings’. One might then expect that this would give a functor to Set extending your Ent functor.

Early morning rushed thoughts…

Posted by: Simon Willerton on November 25, 2021 8:52 AM | Permalink | Reply to this

Re: Compositional Thermostatics

Okay, so now some late night thoughts!

I believe that I have a proof (based on your proof part (1) of your Theorem 21) that if f:X×Yf\colon X\times Y\to \mathbb{R} and g:Y×Zg\colon Y\times Z\to \mathbb{R} are concave, then the composite or convolution gf:X×Z¯g\circ f\colon X\times Z\to \overline{\mathbb{R}} is concave, where gf(x,z)=sup y(g(y,z)+f(x,y)).g\circ f(x,z)= \sup_y(g(y,z) + f(x,y)). The proof that I’ve got should also work for the case of ff or gg taking infinite values provided that c λc_\lambda is appropriately linear and ordered on ¯\overline{\mathbb{R}}, namely c λ(a,b)+c λ(c,d)=c λ(a+c,b+d)foralla,b,c,d¯c_\lambda(a, b)+c_\lambda(c,d)= c_\lambda(a+c, b+d) \quad for\ all a,b,c,d\in \overline{\mathbb{R}} and c λ(a,b)c λ(c,d)wheneverac,bd.c_\lambda(a,b)\ge c_\lambda(c,d)\quad whenever\quad a\ge c, b\ge d. [I’m sure these should follow from the arithmetic you’ve defined, but I’m falling asleep now.]

You show how to get a concave function R *S:Z¯R_\ast S\colon Z\to \overline{\mathbb{R}} from a concave function S:Y¯S\colon Y\to \overline{\mathbb{R}} and a convex subset RY×ZR\in Y\times Z. We have R *S=I RSR_\ast S = I_R\circ S where I R:Y×Z¯I_R\colon Y\times Z\to \overline{\mathbb{R}} is the indicator function on RR and where SS is thought of as being defined on ×Y{\star} \times Y.

This all has the feel of profunctors enriched over ¯\overline{\mathbb{R}} (which is the opposite category of what I usually call ¯\overline{\mathbb{R}}.)

Posted by: Simon Willerton on November 25, 2021 11:09 PM | Permalink | Reply to this

Re: Compositional Thermostatics

For some reason this makes me think of the Lagrangian relations stuff that John has also done.

Posted by: David Roberts on November 26, 2021 12:05 AM | Permalink | Reply to this

Re: Compositional Thermostatics

Since convex linear combinations are “things parametrised by the 1-simplex”, I wonder if there’s a nice simplicial rephrasing of some of these definitions… (just an idle thought, without any effort put into thinking about it any further!)

Posted by: Tim Hosgood on November 25, 2021 11:04 AM | Permalink | Reply to this

Re: Compositional Thermostatics

Convex linear combinations of 2 elements of a convex space are indeed parametrized by the 1-simplex. But what’s really exciting is that convex linear combinations of nn elements of a convex space are parametrized by the (n1)(n-1)-simplex. This fact is captured by an operad P\mathbf{P} whose set of nn-ary operations is the (n1)(n-1)-simplex!

Another way to think about P\mathbf{P} is that its nn-ary operations are probability distributions on the nn-element set.

Any convex space gives an algebra of the operad P\mathbf{P}. But there are others, too, since operads are unable to express laws that involve duplication or deletion of variables. Two of the laws I wrote for convex spaces involve duplication or deletion of variables. They are the ‘boring’ laws:

1) c 1(x,y)=xc_1(x,y) = x

and

2) c λ(x,x)=xc_\lambda(x,x) = x

I believed that algebras of the operad P\mathbf{P} are precisely sets with operations c λc_\lambda obeying laws 3) and 4).

These are algebras in the category of sets. We can also look at algebras in other categories, and Tom did this here:

He shows that Shannon entropy inevitably pops out of the study of P\mathbf{P}!

So, it seems inevitable that thermodynamics and the study of entropy will be understood better when we think harder about this stuff.

Posted by: John Baez on November 26, 2021 2:01 PM | Permalink | Reply to this

Re: Compositional Thermostatics

Thanks for the plug, John. That stuff is written up properly as Chapter 12, “The categorical origins of entropy”, of my new book, Entropy and Diversity.

Of course, you may prefer an improper write-up to a proper one.

Posted by: Tom Leinster on November 26, 2021 3:25 PM | Permalink | Reply to this

Post a New Comment