Skip to the Main Content

Note:These pages make extensive use of the latest XHTML and CSS Standards. They ought to look great in any standards-compliant modern browser. Unfortunately, they will probably look horrible in older browsers, like Netscape 4.x and IE 4.x. Moreover, many posts use MathML, which is, currently only supported in Mozilla. My best suggestion (and you will thank me when surfing an ever-increasing number of sites on the web which have been crafted to use the new standards) is to upgrade to the latest version of your browser. If that's not possible, consider moving to the Standards-compliant and open-source Mozilla browser.

September 29, 2003

Statistical Innumeracy

The science fiction writer H G Wells predicted that in modern technological societies statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write. How far have we got, a hundred or so years later? A glance at the literature shows a shocking lack of statistical understanding of the outcomes of modern technologies, from standard screening tests for HIV infection to DNA evidence…
— Gigerenzer & Edwards, British Medical Journal 327 (2003), 741–744

1) A woman goes to her doctor’s office for a mammogram. She is told that

  1. The incidence of breast cancer is 0.8%.
  2. The “sensitivity” of the mammogram is 90%. (If she has breast cancer, the probability is 90% that it will show up on the mammogram.)
  3. The “false positive” rate is 7%. (If the woman doesn’t have breast cancer, there’s a 7% chance the mammogram will nonetheless yield a positive result.)

The mammogram comes back “positive”. What is the probability that the woman has breast cancer?

  1. 1%
  2. 7%
  3. 9%
  4. 15%
  5. 24%
  6. 47%
  7. 63%
  8. 83%
  9. 90%

In a recent study, a stunning majority of doctors tested (22/24) could not answer this question correctly.

The abstract to the paper

Bad presentation of medical statistics such as the risks associated with a particular intervention can lead to patients making poor decisions on treatment. Particularly confusing are single event probabilities, conditional probabilities (such as sensitivity and specificity), and relative risks. How can doctors improve the presentation of statistical information so that patients can make well informed decisions?

if, anything, soft-pedals the issue, for it seems that a shocking proportion of doctors themselves don’t understand garden-variety medical statistics.

Matters improved considerably, when the data was presented in terms of “natural”, rather than conditional probabilities (11/24 doctors in the “control group” got the above question right, when it was thus reframed). But that’s not the way this sort of data is conventionally presented to medical practitioners (let alone to patients).

The authors would like to believe that — if one could only “frame” the information in the right way, all would be well with the world. But I don’t see it.

2) Say various factors put the woman in a high risk group for breast cancer. Rather than the 0.8% risk faced by the general population, she faces a 2.4% risk of breast cancer. If the test comes back positive, what is the probability now that she has breast cancer?

More to the point, how do you even decide whether to administer a test or perform a procedure in the first place, without having to grapple with these “icky” statistical considerations. (Positive test results may require invasive/expensive followups, procedures carry their own risks, …)

Maybe patients don’t need to understand this stuff (though that makes a mockery of “informed consent”), but doctors surely do.

And it scares the bejeesus out of me that they, apparently, don’t.

Tip 'o the hat to Chris Bertram at Crooked Timber for making my day.

Oh, yeah, in case you were wondering, the answer to 1) is “c” and the answer to 2) is “e”. But you probably knew that … ;-)

Posted by distler at September 29, 2003 11:11 PM

TrackBack URL for this Entry:   http://golem.ph.utexas.edu/cgi-bin/MT-3.0/dxy-tb.fcgi/229

1 Comment & 0 Trackbacks

Re: Statistical Innumeracy

I couldn’t agree more. We make such irrational decisions when such great data is available. Your example about breast cancer is a good one. There are many other examples around fear too. What should a person worry about more… their plane crashing or drowning in a pool in their backyard? Clearly people worry about the former but the latter has a much hight probability of happening.

-Jim

Posted by: Jim Lifter on January 15, 2011 3:34 PM | Permalink | Reply to this

Post a New Comment