Analysis of Variance – ANOVA

While T-tests are great for small datasets, often you will have many variables at play, and more than two experimental groups. If you run several T-tests over and over again, you would need to run several post-hoc tests. After all, if you run a lot of T-tests, it becomes more likely you find a bogus outcome.

Luckily, there are tests that allow you to compare three or more different groups to each other. Analysis of Variance (ANOVA) testing is one of the most popular ways of exploring your dataset. Essentially, you want to see if the response to a specific factor or group of factors differs between groups.

There is a few assumptions that your data has to meet in order to use this test:

1. The population from which samples are drawn should be normally distributed.
2. The samples should be independent of each other.
3. The variance among the groups should be approximately equal.

The test gives you an F-statistic, which relates to the variance of the different groups. It compares the variability between samples and within the samples themselves. When there is no real difference between the groups, the F-statistic will be close to 1.

One Way ANOVA

A one way ANOVA is used when comparing more than two groups with multiple variables. It evaluates the responses to one independent variable at a time. This might be a relationship between acceleration time and top speed, measured between different brands of cars.

Another type of One Way ANOVA involves repeated measures. The independent variable here has different levels or related groups, for example, Pre-Intervention and Post-Intervention. Using this test will look to see if there are differences in the trajectories of individuals across three or more independent groups.

Two Way ANOVA

Another variation is the Two Way ANOVA, where there are two independent variables that are being tested. You might want to measure the performance of students in different classes by looking at both their baseline stress/cortisol levels as well as their blood pressure.

Mixed ANOVA

What happens when you combine repeated measures with a Two Way ANOVA? You end up with a fun new variation of a statistical test. The Mixed ANOVA will let you take into account repeated measures of more than one independent variable. Here is an example of where this test would be used:

Outcome – Perceived stress levels

Independent Variables

1. Stress levels before and after six weeks of therapy
2. Exercise levels at the start of the study and six weeks later

Groups – Statistics students, Psychology students and Nursing students

Interpreting an ANOVA

Once you conduct the test, you will receive a table with an F-value and one or more P-values. In an ANOVA, if the F-Statistic is large than the F-Critical value for n degrees of freedom, then there are significant differences within one or more of your comparisons.

As usual, when your P-value is less than your alpha, you can reject the null hypothesis. With multiple factors and comparisons, you may only have one comparison that shows a significant difference. Here, post-hoc testing is applied to account for the amount of statistical tests that are conducted. If the value remains significant afterwards, then you can claim a significant difference between those particular groups.

Takeaway

The ANOVA family of statistical tests helps you test differences between more than two different groups. It also supports repeated measures of the same independent variables or factors, making it an asset for any pre-/post- intervention comparison. While the ANOVA is useful, make sure to plot the data and visually assess the means, to make sure that your results make sense.