Progic III
Posted by David Corfield
So we’ve seen, in this thread that, even when we work with nice basic finite sets, the probability monad doesn’t get along too well with logical structure, there being none of that pleasant adjointness between categories of predicates over sets.
However, the probability monad, , does come along with other structure of its own. In particular is a Riemannian manifold, with the Fisher information metric, which is crucially invariant under reparameterization. Recall that this space is composed of maps , i.e., probability distributions over . The Fisher information metric (see, e.g., section 4 of Guy Lebanon’s thesis) takes on a simple form for finite , (p. 21).
Then there’s all sorts of fun you can have with geodesics and (non-metric) connections in the subject called information geometry, which, if you want to understand it, it seems that it helps to be Japanese.
But how about ? Well sticking with finite sets, Lebanon showed that, broadly speaking, the only ‘sensible’ metric to put on this space of conditional distributions, a product of simplices, is the product Fisher information metric (see p. 22 of his thesis). We have then in particular that the space of probabilistic predicates on , , is a Riemannian manifold.
Now, a common thing to do with the space of distributions on is to look at subspaces satisfying various constraints. So we might have a function , and look at distributions, , which satisfy , for some constant .
[Aside: That composition could have been taken as happening in the Kleisli category, if we interpret the function as a probability distribution on conditional on , which for each in is all concentrated at a single real value, and then we take the mean of the ensuing distribution over . So I suppose could have been any arrow in .]
Anyway, these submanifolds of have some ‘logical’ structure to them. The submanifold satisfying all of a set of constraints is the same as the intersection of the set of submanifolds satisfying at least one of the constraints.
Final question, is there anything interesting to say about what’s happening geometrically in the space of probabilistic predicates, ?
Re: Progic III
David wrote:
Both probability theory and logic are much too good to ‘not get along well’ with each other. They get along fine. It’s merely our job to discover how.
In such fundamental mathematical subjects, nothing can be less than perfect! Any apparent imperfection is merely an imperfection in our own understanding. If we follow the tao, all will be well.
But how do we find the tao? I still think it’s a great project to understand precisely how the probability monad gets along with the ‘logical’ operations on the category of sets (or Polish spaces, or whatever): products, coproducts, more general limits and colimits, exponentials, etcetera.
It would also be nice, if you plan to use Polish spaces for your probability monad, to determine how good the category of Polish spaces actually is. I can’t tell if this category is really ‘right’ or just a sloppy stopgap measure: the first thing somebody happened to try.
For example, is the category of Polish spaces cartesian closed? That is, roughly speaking: does the space of maps between Polish spaces become a Polish space in a nice way? If not, we should switch to something better.
I want to work in a category that’s cartesian closed. In other words — for those who don’t grok this jargon — given spaces and , I want to have a space of maps from to :
with a natural isomorphism
just like we’re used to in the category of sets.
Furthermore, I want a probability measure on this space of maps to give map from to probability measures on . In other words, I want a map
Doesn’t that seem sensible?
I don’t know if this map wants to be an isomorphism — I could figure it out in five minutes, but I’ll let someone else. This map should make some diagrams commute… but I don’t know what those diagrams are. Someone who has thought about monads on cartesian closed categories might know.
As mentioned before, I also want a map
Of course this map should not be an isomorphism, since only product measures are in its range. I bet this map will make into a symmetric monoidal monad.
Finally (for now), what about
This one seems sort of funny. There’s not one best map
Instead, there’s one natural map for each : given a probability measure on and a probability measure on , you can flip a -weighted coin to pick either or , then randomly pick an element from the space you’ve chosen.
All these structures should fit together into some nice entity — which may or may not have been given a name and studied to death.