Chapter 10

Brief Overview

• The simple experiment's purpose
• The simple experiment's terminology
• Common misconceptions about
  • Random assignment
  • Independence
  • Statistical significance
  • Null results
• Type 1 and Type 2 errors
• Designing the simple experiment
  • How statistical considerations affect the design of the simple experiment
  • How nonstatistical considerations affect the design of the simple experiment
  • Resolving conflicts between statistical and nonstatistical considerations when designing the simple experiment
• The t test
  • The general logic behind the t test
  • How to use the t test to analyze data from the simple experiment
  • Assumptions of the t test
    • Chances of meeting each assumption
    • Consequences of violating each assumption
• Questions raised by null results
• Questions raised by significant results

Pictorial Summary

Although we carefully explain all the terminology associated with the simple experiment, we stress two key aspects:

Not only do we explain that the need for random assignment limits what kinds of variables we can study in an experiment, but we also explain that the need for independence affects how we conduct experiments.

Tables 10-1 (p. 290) and 10-2 (p. 291) and Figure 10-1 (p. 291) reiterate the key points about interpreting statistically significant as well as null results:

We then go into more detail about assessing as well as reducing the risks of both Type 1 and Type 2 errors. The discussion of reducing Type 2 errors (see Figure 10-2) leads to discussing how statistics affects the design of the simple experiment, a discussion that is summarized in Table 10-4. We then discuss conflicts between the goal of statistical power and other research goals (see Table 10-5). Next, we lay out the logic behind statistical analysis of the simple experiment. Specifically, we

• introduce the concepts of between and within group variability,
• explain that between groups variability can be due to treatment effects and random error
• show that within groups variability is due to random error
• show that significant results are most likely to occur with more between group variability, larger sample sizes, and lower within group variability

Finally, we point out that, no matter what the results of a simple experiment, questions should be raised. For example, if the results are not significant,

If, on the other hand, the results are significant,

Back to Chapter 10 Menu

Back to Research Design Explained Home Page