LECTURE 13.1


I. Logic of the matched pairs design

A. Completely randomized between-subjects designs have internal validity, but between-subjects differences may obscure the true treatment effect. That is, within group variability can give us such a large error term that even a respectable difference between groups will not be found statistically significant.

B. Matching doesn't eliminate differences between participants,

C. However, it does reduce those differences

D. If we match and then use random assignment, we have

1. Internal validity (due to random assignment)

2. Power because matching leaves random assignment with less between-subjects error to balance out. Consequently, it is less likely that random error will hide a treatment effect. For example, suppose that we use a simple experiment and a matched pairs experiment. Both find a difference of 6 units between the treatment and no-treatment groups. However, suppose that the error term in the simple experiment is 6, whereas the error term in the matched pairs experiment is only 1. Then, in the simple experiment, t = 1 (because 6/6 = 1). However, in the matched pairs experiment, t = 6 (because 6/1 = 6).

II. The matched pairs and external validity

A. May have to eliminate participants that can't be matched

B. Results may only generalize to those who have been pretested then received treatment

III. The matched pairs may have poor construct validity if the matching variable sensitizes participants to what the experiment is about

IV. The within-subjects design: The ultimate in matching

A. Power

1. Get more than one observation per participant

2. Avoid problems due to between-subjects variability

B. External validity

1. Results may only generalize to individuals who get multiple treatments

C. Construct validity may be jeopardized by sensitizing the participant to what the study is about

D. Internal validity is threatened by order effects, such as

1. Practice effects:

Ex: practice on a task may improve performance

2. Fatigue effects: In later trials, participants performance may worsen because they are tired or bored.

3. Carry-over effects

Ex: Drug studies, exertion studies, studies in which participants learn a new strategy and keep using it.

4. Sensitization: Participants figuring out, especially by the later trials, what the experiment is about.

VI. Approaching the order problem

A. Randomize order of treatments (but random orders may not perfectly balance out order effects. For example, you may have 8 A-B orders and 2 B-A orders.

B. Counterbalance order and then randomly assign participants to counterbalanced order so that half the participants get A-B, and half get B-A.

1. Works best if order effects are consistent

2. Interpreting a counterbalanced design can be a challenge

A. Main effect of Treatment =

B. Main effect of Counterbalancing Sequence =

C. Interaction of Treatment X Counterbalancing (In other words, the effect of order [first trial vs. Second trial)=

Ex:


Average score on trial
where participants
received Treatment 1
Average score on trial
where participants
received Treatment 2
Group of participants
getting the CB sequence
Treatment 1, then
Treatment 2
17 5
Group of participants
getting the CB sequence
Treatment 2, then
Treatment 1
5 17

C. Take steps to reduce practice, fatigue, carry-over, and sensitization

1. Extensive pre-experimental practice to prevent practice effects

2. Short, non-fatiguing experiments to prevent fatigue effects.

3. Space treatments far apart to prevent carry-over

4. Use unobtrusive measures and/or introduce manipulation so gradually that your participants don't notice to prevent sensitization

D. Study order effects because you're interested in order (ex: In memory experiments, you may be interested in effects of practice or interference effects).

E. Avoid the problem of order effects by using a totally between subjects design


Back to Chapter 13 Main Menu