How Reliable Are Your Measures? Introducing Cronbach’s Alpha

I’m sure you recall the stress of Leaving Certs, those dreaded standardized tests. When you become a superstar researcher, you might discover something interesting and you’ll no doubt make a questionnaire to measure it. We run into the question of validation: how do you know that your scale or test is reliable?

Cronbach’s Alpha

Let’s run a hypothetical situation, aliens land on Earth. They look much different than anything you can possibly imagine. Nonetheless, these weird beings are great fun to be around. Over a few pints in the pub, your space friends tell you about a caffeinated drink on their planet. This drink is supposed to boost attention and awareness.

Hearing that you’re a researcher, the aliens ask for help measuring the effects of their caffeinated drink. Luckily, the aliens have data from a lot of different testing scales for measuring these effects. How would you determine if any of these scales are useful?

Enter Cronbach’s alpha. If there’s a scale that measures a specific concept, that means that the questions and tasks included in the scale all measure something similar. Logically, you’d find a lot of covariance between these different features on the scale, if they were measuring the same concept. Cronbach’s alpha gives you a value between 0 and 1.

If Cronbach’s alpha is greater than 0.70, it implies covariance between items in the scale and internal consistency. You present the aliens with the formula to calculate it:

      α =  (number of items * average interitem co-variance)

             (average variance + (N-1) * average interitem co-variance)

Why It’s Important

We’ve illustrated how we calculate this measure but let’s talk about why we should care about it. For instance, there’s currently a replication crisis in psychology where many experimental results fail to repeat when retested. Many experiments are plagued by poor methodology or statistics. Sometimes, the problem is with the psychometric tests being used. In part, some of the ways that psychological constructs are operationalized, described, and measured might be unreliable.

You, yes you, may stick through psychology and enter the research setting. You might opt for an industry or data analysis position instead. Nonetheless, determining whether your measurements are reliable will remain important no matter your career. It is going to be up to you, to use critical thinking and statistics to make sure the tests you use to measure different constructs make sense.