Skip to the Main Content

Note:These pages make extensive use of the latest XHTML and CSS Standards. They ought to look great in any standards-compliant modern browser. Unfortunately, they will probably look horrible in older browsers, like Netscape 4.x and IE 4.x. Moreover, many posts use MathML, which is, currently only supported in Mozilla. My best suggestion (and you will thank me when surfing an ever-increasing number of sites on the web which have been crafted to use the new standards) is to upgrade to the latest version of your browser. If that's not possible, consider moving to the Standards-compliant and open-source Mozilla browser.

November 8, 2021

Causality in Machine Learning

Posted by David Corfield

Back when we started the Café in 2006, I was working as a philosopher embedded with a machine learning group in the Max Planck Institute in Tübingen. Here I am reporting on my contribution to a NIPS workshop, held amongst the mountains of Whistler, on how one may still be able to learn when the distributions from which data is drawn for training and testing purposes differ. My proposal was that background knowledge, much of it causal, had to be deployed. It turns out that a video of the talk is still available – links to this and the resulting book chapter, Projection and Projectability, are here.

I was reminded of this work recently after seeing the strides taken by the machine learning community to integrate causal graphical models with their statistical techniques in Towards Causal Representation Learning and Causality for Machine Learning. Who knows? Perhaps my talk, which was after all addressed to some of these people, sowed a seed.

But another seed I was trying to sow around that time was Category Theory in Machine Learning (see also posts of mine from around that time on, e.g., kernels, infinite-dimensional exponential families, and probability theory). And I see things are happening on this front too, much summarised in Category Theory in Machine Learning.

I’m left wondering about the role of philosophy. Are we better advised sticking to the ‘making sense of what’s happened’ part of our jobs, often addressed to each other, or is there a place for a ‘you people might want to take a look at this’ approach, addressed outside?

Posted at November 8, 2021 10:15 AM UTC

TrackBack URL for this Entry:   https://golem.ph.utexas.edu/cgi-bin/MT-3.0/dxy-tb.fcgi/3365

4 Comments & 0 Trackbacks

Boryslaw

It seems to me that the central topic of this post is exactly what in causal inference is known as transportability, see for example Pearl, J., and Bareinboim, E. (2011) “Transportability of Causal and Statistical Relations: A Formal Approach”

Posted by: Boryslaw Paulewicz on November 8, 2021 7:02 PM | Permalink | Reply to this

Re: Boryslaw

Yes, this is exactly it. Thanks.

I see the authors refer to Amos Storkey’s contribution to the same volume in which my article appears. Amos was speaking at the same workshop. His was the only other paper there to raise the matter of causality.

Hmm, 240 citations to my 0.

Posted by: David Corfield on November 9, 2021 8:00 AM | Permalink | Reply to this

Re: Causality in Machine Learning

David wrote:

Are we better advised sticking to the ‘making sense of what’s happened’ part of our jobs, often addressed to each other….

That would be tragic, making philosophy into almost an ‘epiphenomenon’ in the history of thought:

“An epiphenomenon can be an effect of primary phenomena, but cannot affect a primary phenomenon.”

But I’ve almost despaired of philosophy helping solve important problems in science — in part because scientists don’t listen much to philosophers, and in part because many philosophers seem content to analyze what’s already happened, instead of joining the fray of science as Aristotle, Descartes and Leibniz did. Of course there are exceptions, like you — and your frustrations seem to exemplify the problems I’m talking about.

Posted by: John Baez on November 26, 2021 2:12 PM | Permalink | Reply to this

Re: Causality in Machine Learning

Some areas have been much better than others for philosophy-science exchange. For instance, there’s been good collaboration between philosophy and some parts of biology.

The interesting thing about the rise of applied category theory is that it’s treating topics considered central by philosophers, but with noticeably different tools. It needs young philosophers to see what advantages they might gain.

Posted by: David Corfield on November 27, 2021 4:04 PM | Permalink | Reply to this

Post a New Comment