Quasi-experimental Research Designs

True Quasi-experimental Research Designs – in which a treatment or stimulus is administered to only one of two groups whose members were randomly assigned – are considered the gold standard in assessing causal hypotheses.  True experiments require researchers to exert a great deal of control over all aspects of the design, which in turn allows strong statements to be made about causal relationships.  In many situations, especially those involving human subjects, it is simply not possible for researchers to exert the level of control necessary for a true experiment.  For example, it may be unethical to expose subjects to a stimulus which the researcher knows may cause harm.  In addition, researchers are often interested in processes that are too complex or lengthy to be administered in an experimental setting.  Quasi-experimental designs relax some of the key requirements of true experiments, making them more practical to implement in many cases but also reducing the strength of the causal claims that can be made.

True experiments require that subjects be randomly assigned to the treatment or control group.  Random assignment ensures that any characteristics of the subjects which may be associated with the outcome of interest will be distributed throughout the two groups according to the laws of probability.  Often it is not possible for researchers to randomly assign subjects to groups, for either practical or ethical reasons.  Quasi-experimental research designs therefore use alternative ways of assigning subjects to the treatment and control groups.

The most common subset of quasi-experimental research designs are the nonequivalent control group designs.  In one implementation of this design, subjects in the control group are intentionally matched by the researcher to subjects in the treatment group on characteristics which might be associated with the outcome of interest.  This matching can be done at the individual level, resulting in a one-to-one match of individuals in the two groups.  Another approach is aggregate matching, in which researchers select a control group with the same general composition of relevant characteristics (for example, the same proportion of females and the same age distribution) as the treatment group.  These approaches are considered quasi-experimental due to the fact that assignment of subjects to groups is intentional and not random.  Another common approach to this type of quasi-experimental research design is the use of existing groups.  For example, a comparison could be made between students in two classrooms, with the stimulus administered in only one classroom.

Some quasi-experimental research designs do not include a comparison with a control group at all.  Known as before-and-after, pre-test/post-test, or pre-experimental designs, these quasi-experimental approach designs expose all subjects to the treatment or stimulus.  The comparison in these designs comes from examining subjects’ values on the outcome of interest prior to and after the exposure.  If post-treatment values differ significantly from pre-treatment values, a case can be made that the treatment was the cause of the change.

Another quasi-experimental approach involves time-series data, in which researchers observe one group of subjects repeatedly both before and after the administration of the treatment.  This can be done in a controlled experimental setting, but this design also lends itself well to a more naturalistic setting in which data are commonly collected on a group of subjects and researchers are interested in the effects of some treatment or intervention which they did not experimentally apply.  For example, researchers might examine the yearly test scores of students at a given school for several years both before and after the implementation of an extended school day; in this situation the yearly tests scores represent the time-series data and the change to an extended school day is the naturally occurring, quasi-experimental treatment.  This approach is an improvement over the single pre-test/post-test design, which is unable to demonstrate long-term effects.  The time-series data design can be further improved by including a control group which is also examined over time but which does not experience the treatment; such a design is termed a multiple time-series design.

While quasi-experimental designs are often more practical to implement than true experiments, they are more susceptible to threats to internal validity.  Special care must be taken to address validity threats, and the use of additional data to rule out alternate explanations is advised.