### Common Applications

#### Posted by David Corfield

I’ve been reading some of Jorg Lemm’s papers in recent days. He’s written a book - Bayesian Field Theory - which I don’t have access to, but he had written a paper of the same name earlier. In it (page 6, note 1) he remarks that:

statistical field theories, which encompass quantum mechanics and quantum field theory in their Euclidean formulation, are technically similar to a nonparametric Bayesian approach.

It is intriguing to see so many constructions of mathematical physics - mean field methods, diffusion models, free energy - find a use in learning theory. But what to make of it? If we think it needs an explanation at all, we might say that perhaps it’s telling us that we only have a limited number of tools, so should expect to use them time and again. If we were washed up on a desert island with just a knife in our pocket, we’d find a host of uses for it, with little in common between them, e.g., opening a clam and sharpening a stick.

David Ruelle favoured this kind of explanation about multiple application in “Is our mathematics natural? The case of equilibrium statistical mechanics.” Bull. Amer. Math. Soc. 19, 259-268 (1988). Our minds have a limited repertoire, which explains why mathematicians keep bumping into the same constructions. Closer to this blog, a similar question is why the deeper reaches of number theory (Langlands programme) and quantum field theory (duality) are so closely related. In Mathematics in the 20th Century, Michael Atiyah’s predictions for the 21st century went thus:

What about the 21st century? I have said the 21st century might be the era of quantum mathematics or, if you like, of infinite dimensional mathematics. What could this mean? Quantum mathematics could mean, if we get that far, ‘understanding properly the analysis, geometry, topology, algebra of various non-linear function spaces’, and by ‘understanding properly’ I mean understanding it in such a way as to get quite rigorous proofs of all the beautiful things the physicists have been speculating about.

This work requires generalising the duality between position and momentum in classical mechanics:

This replaces a space by its dual space, and in linear theories that duality is just the Fourier transform. But in non-linear theories, how to replace a Fourier transform is one of the big challenges.

Large parts of mathematics are concerned with how to generalise dualities in nonlinear situations.Physicists seem to be able to do so in a remarkable way in their string theories and in M-theory…understanding those non-linear dualities does seem to be one of the big challenges of the next century as well. (Atiyah 2002: 14-15, my emphasis)

Again, is this just a sign of our limited repertoire? (Perhaps Atiyah might have said that the problem is also how to categorify such dualities.)

A second strain of explanation for multiple application of a piece of mathematics, on the other hand, is that the things it is applied to really are similar. It is no accident that the same tools work in different situations when the tasks are very similar. With regards to commonalities between Bayesian statistics and physics, Edwin Jaynes would favour this latter explanation. Recently this has been expressed by Caticha in The Information Geometry of Space and Time:

The point of view that has been prevalent among scientists is that the laws of physics mirror the laws of nature. The reflection might be imperfect, a mere approximation to the real thing, but it is a reflection nonetheless. The connection between physics and nature could, however, be less direct. The laws of physics could be mere rules for processing information about nature. If this second point of view turns out to be correct one would expect many aspects of physics to mirror the structure of theories of inference. Indeed, it should be possible to derive the “laws of physics” appropriate to a certain problem by applying standard rules of inference to the information that happens to be relevant to the problem at hand.

Noting that statistical mechanics and quantum mechanics can be largely constructed by considering them as ways of manipulating information, Caticha goes on to take on general relativity.

Now, John Baez raises an interesting example of commonality of structure in this comment, between natural selection and Bayesian inference. I could imagine explanations by both of the strains above.

- The structure of Bayes’ theorem (which doesn’t require you to be a Bayesian to use) is a very simple one relevant in many combinatorial situations, which is how we like to think about the world.
- Evolution is a kind of learning.

## Re: Common Applications

Interesting observartions.

A tiny comment:

I think everybody will agree that the general pattern of statistical mechanics is indeed about more about inference than about nature per se. But at some point you want to apply all this to a particular case. Usually this amounts to specifying a Hamiltonian function.

And the precise details of

thatfunction is what encodes information about nature.So there is a bit of information about nature - encoded in a Hamiltonian - and then there are means to extract certain parts of that information (entropy maximization, etc.).

Interestingly, while quantum mechanics is in a way nothing but statistical mechanics analytically continued to the complex plane, we usually tend to regard not just the Hamiltonian in quantum mechanics as encoding information about nature, but also the rest of the formalism.

Whether that “rest of the formalism” is really just a manifestation of our thinking or a genuine aspect of nature is hotly debated in all those discussions concerning the “interpretation of quantum mechanics”.