Brief Summary of Chapter 14

To establish that a treatment causes an effect, you must

  1. introduce a treatment
  2. observe a change in behavior
  3. make sure that nothing else besides the treatment could be responsible for the change in behavior

Researchers using randomized experiments rely on statistical tests to argue that it is unlikely that anything other than treatment caused the change in behavior. Single-n researchers try to prevent factors that could change the behavior from varying. Studying participants under highly controlled laboratory conditions, for example, reduces the risk of the change being due to history. Maturation, however, still might be a problem. Therefore, single-n researchers prefer A (no treatment)-B (treatment)-A (no-treatment) designs to A (no-treatment)-B (treatment) designs.

Quasi-experimenters can't keeping all of the critical factors constant. Therefore, they must rely on their wits to rule out Campbell and Stanley's eight threats to internal validity. For example, if they used a before-after design, they would have to worry that the change in behavior could be due to:

Fortunately, by adding several more times of measurement to a before-after design, quasi-experimenters can better estimate the effects of these threats. Thus, if there are only modest changes in behavior in the weeks before the treatment is administered, but a large change in behavior right after the treatment is administered, then the quasi-experimenter would be relatively confident that the change was due to the treatment. However, it is still possible that the difference was not due to the treatment. A change in the environment (history), rather than the treatment, could be responsible for the sudden change. For example, suppose you are trying to change people's attitudes toward government regulation of food. Then, the week you administer the treatment, there is an outbreak of food poisoning. The food poisoning outbreak may have more effect than your manipulation. To minimize these problems, you might use a reversal time-series design. In that case, you would get baseline data, introduce the treatment, collect data, withdraw the treatment, and collect data. Your hope would be that when you withdrew the treatment, the behavior would revert back to what it was before you introduced the treatment.

Instead of using a time-series design, you might compare a no-treatment group with a treatment group. A big question with this design is: Do the groups differ now because one group got the treatment--or did the two groups differ to start with? In other words, with a two-group design, you have to worry about selection. It is very hard to eliminate the selection threat. Basically, the problem is that you can't get two identical groups of participants.

Some people naively think that matching will give you identical groups, but those people are failing to realize that:

Back to Chapter 14 Menu