This site is not maintained. Click here for the new website of Richard Dawkins.

The Mind-Reading Salmon: The True Meaning of Statistical Significance

If you want to convince the world that a fish can sense your emotions, only one statistical measure will suffice: the p-value.

The p-value is an all-purpose measure that scientists often use to determine whether or not an experimental result is “statistically significant.” Unfortunately, sometimes the test does not work as advertised, and researchers imbue an observation with great significance when in fact it might be a worthless fluke.

Say you’ve performed a scientific experiment testing a new heart attack drug against a placebo. At the end of the trial, you compare the two groups. Lo and behold, the patients who took the drug had fewer heart attacks than those who took the placebo. Success! The drug works!

Well, maybe not. There is a 50 percent chance that even if the drug is completely ineffective, patients taking it will do better than those taking the placebo. (After all, one group has to do better than the other; it’s a toss-up whether the drug group or placebo group will come up on top.)

The p-value puts a number on the effects of randomness. It is the probability of seeing a positive experimental outcome even if your hypothesis is wrong. A long-standing convention in many scientific fields is that any result with a p-value below 0.05 is deemed statistically significant. An arbitrary convention, it is often the wrong one. When you make a comparison of an ineffective drug to a placebo, you will typically get a statistically significant result one time out of 20. And if you make 20 such comparisons in a scientific paper, on average, you will get one signif­icant result with a p-value less than 0.05—even when the drug does not work.
Read more

TAGGED: MATH


RELATED CONTENT

Pierre de Fermat's Last Theorem...

Nathan Green - guardian.co.uk 19 Comments

The Mathematics of Changing Your Mind

JOHN ALLEN PAULOS - The New York Times 30 Comments

The Unreasonable Beauty of Mathematics...

George Musser - Scientific American 15 Comments

Science, Culture And The Google Ngram...

Ajita Kamal - Nirmukta 15 Comments

Numberplay: Rare Coincidences Are Very...

PRADEEP MUTALIK - The New York Times 28 Comments

Richard Dawkins at the 'Genius of...

May 26 2010 at the Science Museum in... 13 Comments

MORE

MORE BY CHARLES SEIFE

MORE

Comments

Comment RSS Feed

Please sign in or register to comment