Conduct and Interpret a Repeated Measures ANOVA

What is the Repeated Measures ANOVA?

The repeated measures ANOVA is a member of the ANOVA family.  ANOVA is short for ANalysis Of VAriance.  All ANOVAs compare one or more mean scores with each other; they are tests for the difference in mean scores.  The repeated measures ANOVA compares means across one or more variables that are based on repeated observations.  A repeated measures ANOVA model can also include zero or more independent variables.  Again, a repeated measures ANOVA has at least 1 dependent variable that has more than one observation.

Get a Jump Start on Your Quantitative Results Chapter

quant-results



Example:

A research team wants to test the user acceptance of a new online travel booking tool.  The team conducts a study where they assign 30 randomly chosen people into two groups.  One group uses the new system and another group acts as a control group and books its travel via phone.  The team measures the user acceptance of the system as the behavioral intention to use the system in the first 4 weeks after it went live.  Since user acceptance is a latent behavioral construct the researchers measure it with three items – ease of use, perceived usefulness, and effort to use.

The repeated measures ANOVA is an ‘analysis of dependencies’.  It is referred to as such because it is a test to prove an assumed cause-effect relationship between the independent variable(s), if any, and the dependent variable(s).

When faced with a question similar to the one in our example, you could also try to run 4 MANOVAs, testing the influence of the independent variables on each of the observations of the four weeks.  Running multiple ANOVAs, however, does not account for individual differences in baselines of the participants of the study.

The repeated measures ANOVA is similar to the dependent sample T-Test, because it also compares the mean scores of one group to another group on different observations.  It is necessary for the repeated measures ANOVA for the cases in one observation to be directly linked with the cases in all other observations.  This automatically happens when repeated measures are taken, or when analyzing similar units or comparable specimen.

The pairing of observations or making repeated measurements are very common when conducting experiments or making observations with time lags.  Pairing the measured data points is typically done in order to exclude any cofounding or hidden factors (cf.  partial correlation).  It is also often used to account for individual differences in the baselines, such as pre-existing conditions in clinical research.  Consider for example a drug trial where the participants have individual differences that might have an impact on the outcome of the trial.  The typical drug trial splits all participants into a control and the treatment group and measures the effect of the drug in month 1 -18.  The repeated measures ANOVA can correct for the individual differences or baselines.  The baseline differences that might have an effect on the outcome could be typical parameter like blood pressure, age, or gender.  Thus the repeated measures ANOVA analyzes the effect of the drug while excluding the influence of different baseline levels of health when the trial began.

Since the pairing is explicitly defined and thus new information added to the data, paired data can always be analyzed with a regular ANOVA as well, but not vice versa.  The baseline differences, however, will not be accounted for.

A typical guideline to determine whether the repeated measures ANOVA is the right test is to answer the following three questions:

  • Is there a direct relationship between each pair of observations, e.g., before vs.  after scores on the same subject?
  • Are the observations of the data points definitely not random (i.e., they must not be a randomly selected specimen of the same population)?
  • Do all observations have to have the same number of data points?

If the answer is yes to all three of these questions the dependent sample t-test is the right test.  If not, use the ANOVA or the t-test.  In statistical terms the repeated measures ANOVA requires that the within-group variation, which is a source of measurement errors, can be identified and excluded from the analysis.

The Repeated Measures ANOVA in SPSS

Let us return to our aptitude test question in consideration of the repeated measures ANOVA.  The question being: “Is there a difference between the five repeated aptitude tests between students who passed the exam and the students who failed the exam?” Since we ran the aptitude tests multiple times with the students these are considered repeated measurements.  The repeated measures ANOVA uses the GLM module of SPSS, like the factorial ANOVAs, MANOVAs, and MANCOVAs.

The repeated measures ANOVA can be found in SPSS in the menu Analyze/General Linear Model/Repeated Measures…

The dialog box that opens on the click is different than the GLM module you might know from the MANOVA.  Before specifying the model we need to group the repeated measures.

We specify the repeated measures by creating a within-subject factor.  It is called within-subject factor of our repeated measures ANOVA because it represents the different observations of one subject (so the measures are made within one single case).  We measured the aptitude on five longitudinal data points.  Therefore we have five levels of the within-subject factor.  If we just want to test whether the data differs significantly over time we are done after we created and added the factor Aptitude_Tests(5).

The next dialog box allows us to specify the repeated measures ANOVA.  First we need to add the five observation points to the within-subject variables simply select the five aptitude test points and click on the arrow pointing towards the list of within-subject variables.  In a more complex example we could also include additional dependent variables into the analysis.  Plus we can add treatment/grouping variables to the repeated measures ANOVA, in such a case the grouping variable would be added as a between-subject factor.

Since our example does not have an independent variable the post hoc tests and contrasts are not needed to compare individual differences between levels of the between-subject factor.  We also go with the default option of the full factorial model (in the Model… dialog box).  If you were to conduct a post hoc test, SPSS would run a couple of pairwise dependent samples t-tests.  We only add some useful statistics to the repeated measures ANOVA output in the Options… dialog.

Technically we only need the Levene test for homoscedasticity when we would include at least one independent variable in the sample.  However it is checked here out of habit so that we don't forget to select it for the other GLM procedures we run.

It is also quite useful to include the descriptive statistics, because we have not yet compared the longitudinal development of the five administered aptitude tests.