Scientists Say: Statistical significance

Scientists use this phrase when they think they have a result that is too different to have happened by accident

Bars on graphs may look different. But are they just an accident? Scientists call differences that probably didn’t occur by chance “statistically significant.”

WDnet/istockphoto

By Bethany Brookshire

December 3, 2018 at 6:30 am

Statistical Significance (noun, “Stah-TISS-tih-cull Sig-NIFF-ih-cance”)

When a scientist talks about the results of their experiment, they might say their finding was “significant.” That’s not because the result will change science (though it might). In research, statistical significance is a phrase that scientists use when the difference they measure is unlikely to have occurred by chance.

A lot of things — in science and in life — happen by accident. Scientists try to make sure that accidents don’t happen. But they can’t prevent them all. Say a scientist is testing a fertilizer to see if it makes plants bigger. They give one group of plants the fertilizer and the other gets nothing but water and sun. But one plant in a greenhouse might get a little more water than another. Another might get slightly more sunlight. If the fertilized plants are taller than the unfertilized plants, how can the scientist be sure the fertilizer was the cause? They can’t. They can only say how likely it was the taller plants could happen by chance.

Usually, statistical significance is defined as a probability. The probability being measured is how likely it is that a difference scientists measured was due to accident. They call this probability a p value. Many scientists accept a p value of 0.05 as statistically significant. That would mean that the results a scientist saw from their experiment would occur by chance only five percent of the time.

But just because a finding is statistically significant does not mean it is meaningful. A scientist might see a statistically significant result in cells in a dish. But it might not mean anything for a whole person’s health. A researcher might see a statistically significant result in a small sample of people. But the difference might disappear when more people are tested. A statistically significant finding can be interesting. But it should always be treated with caution.