Musings

Statistical Innumeracy

The science fiction writer H G Wells predicted that in modern technological societies statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write. How far have we got, a hundred or so years later? A glance at the literature shows a shocking lack of statistical understanding of the outcomes of modern technologies, from standard screening tests for HIV infection to DNA evidence…
— Gigerenzer & Edwards, British Medical Journal 327 (2003), 741–744

1) A woman goes to her doctor’s office for a mammogram. She is told that

1. The incidence of breast cancer is 0.8%.
2. The “sensitivity” of the mammogram is 90%. (If she has breast cancer, the probability is 90% that it will show up on the mammogram.)
3. The “false positive” rate is 7%. (If the woman doesn’t have breast cancer, there’s a 7% chance the mammogram will nonetheless yield a positive result.)

The mammogram comes back “positive”. What is the probability that the woman has breast cancer?

1. 1%
2. 7%
3. 9%
4. 15%
5. 24%
6. 47%
7. 63%
8. 83%
9. 90%

In a recent study, a stunning majority of doctors tested (22/24) could not answer this question correctly.

The abstract to the paper

Bad presentation of medical statistics such as the risks associated with a particular intervention can lead to patients making poor decisions on treatment. Particularly confusing are single event probabilities, conditional probabilities (such as sensitivity and specificity), and relative risks. How can doctors improve the presentation of statistical information so that patients can make well informed decisions?

if, anything, soft-pedals the issue, for it seems that a shocking proportion of doctors themselves don’t understand garden-variety medical statistics.

Matters improved considerably, when the data was presented in terms of “natural”, rather than conditional probabilities (11/24 doctors in the “control group” got the above question right, when it was thus reframed). But that’s not the way this sort of data is conventionally presented to medical practitioners (let alone to patients).

The authors would like to believe that — if one could only “frame” the information in the right way, all would be well with the world. But I don’t see it.

2) Say various factors put the woman in a high risk group for breast cancer. Rather than the 0.8% risk faced by the general population, she faces a 2.4% risk of breast cancer. If the test comes back positive, what is the probability now that she has breast cancer?

More to the point, how do you even decide whether to administer a test or perform a procedure in the first place, without having to grapple with these “icky” statistical considerations. (Positive test results may require invasive/expensive followups, procedures carry their own risks, …)

Maybe patients don’t need to understand this stuff (though that makes a mockery of “informed consent”), but doctors surely do.

And it scares the bejeesus out of me that they, apparently, don’t.

Tip 'o the hat to Chris Bertram at Crooked Timber for making my day.