The Importance of Effect Size

Beginning in 1990, scientists embarked on an epic mission – sequencing the entire human genome. By sequencing our DNA, scientists hoped to unlock the secrets of health, longevity, and disease.

But after sequencing the entire human genome, scientists ran into a problem – interpreting all that data.

As it happens, most genetic variations don’t account for a lot of interindividual differences. One trait for example, results from the interaction of dozens of genetic differences with small effect sizes. What kind of information can we draw from a result with a small effect size?

When Effect Sizes Aren’t Meaningful

Determining whether there is a difference between two groups isn’t enough. We need to understand whether the difference is biologically, clinically or psychologically meaningful.

To do this, we quantify the magnitude of a group difference or the strength of an association.

We might see large population-level studies of nutrition finding meager differences between groups. Let’s look at the theoretical example below.

Drinking 4 cups of coffee reduces mortality with an effect size of 0.98. This means that there is a 2 percent reduction in death. If someone would have a 3 percent chance of mortality in a certain year, a 2 percent reduction would reduce the mortality rate to 2.94 percent.

Depending on your sample size and confidence intervals, you’d need to interpret the meaning of this data. Will getting people to drink more cups of coffee lead to meaningful changes in health outcomes? Probably not.

Small effect sizes might also be artefacts in your study, disappearing as the sample size approaches the population size.

Calculating Effect Sizes

When testing differences between the means of two different groups, we can calculate effect size by using Cohen’s d. Simply, it’s the absolute difference between two means, divided by the standard deviation.

The higher the value, the stronger the effect size. A value of +/- 0.2 is considered small, a value of +/- 0.5 is considered medium and a value of +/- 0.8 is considered a large effect. A negative effect size means that the experimental mean is lower than the control mean.

A positive effect size indicates the experimental mean is larger. Notice that as the standard deviation decreases, the value of Cohen’s d increases. This is because the samples from each of the two groups are more closely clustered to the mean.

For relationships, the coefficient r gives us the strength or magnitude of the effect size. We will discuss r later while exploring the different types of relationship tests.

Takeaway

Effect size gives us more information than the P-value. Instead of telling us whether or not to reject a null hypothesis, an effect size quantifies the magnitude and direction of these differences. They help you determine if your results are psychologically, clinically or biologically meaningful.