THE EXPERIMENT: A Tool for Allowing Us to Determine Whether "A" Causes Changes in "B."
Definition of the simple experiment: a research tool that allows scientists to find out whether a treatment influences (causes) a given behavior or mental characteristic by randomly assigning some participants to get the treatment and other participants to not receive the treatment.
Brief overview of the Simple Experiment
| No treatment group | Average Score |
| Treatment group | Average Score |
| No caffeine group | 15 seconds |
| Caffeine group | 9 seconds |
Why do we do experiments?
To make cause-effect statements.
Two types of hypotheses tested in experiments:
1. Experimental hypothesis: prediction that the
treatment causes an effect. It can be proven wrong.
2. Null hypothesis: prediction that the treatment does not cause an effect. It also can be proven wrong. It cannot, however, be accepted.
Joke illustrating problem of accepting the null hypothesis.
More serious problem that could result from accepting the null hypothesis.
What happens if we disprove the null hypothesis?
(Hint: If the statement "I did not eat the candy bar" is false, what did I do?)
Why do we have two groups?
(Hint: In the Skinner experiment with the rats running the mazes, what could we have concluded if we had only used a caffeine group. That is, what could we have concluded about the effects of caffeine if all we knew was that the rats getting caffeine ran the maze in 9 seconds?)
How can we avoid comparing apples with oranges?
(How do we know that our treatment group and control group were similar before we introduced the treatment?)
Random assignment to treatment involves using a
system where everyone who participates in the study has
an equal chance of being put into the treatment group.
Time to take an ungraded quiz.
What's the problem with random assignment?
Hint: Suppose we get these results:
Experimental Group = 75%
Control Group = 74%
Could these results be due to random assignment creating groups that were slightly different before we introduced the treatment?
How can this problem be solved?
Tests of statistical significance determine if the difference is to big to be due to chance alone
The tests look at two factors:
1. They look at the size of the difference.The bigger the difference between the groups, the more likely the results are to be statistically significant. For example, if the Experimental group averages 95% and the control group averages 45% on our test, that difference would probably be statistically significant. (Intuitively, you do the same thing. If your team gets beat by one point, you point out that the other team was lucky. You don't have to concede that the other team is better. However, if they beat your team by 30 points, you may have to admit that the other team is better).
2. They look at the number of participants.The more participants that are used, the more likely the results are to be statistically significant. (Why? Because if you only have a few participants, the groups might be very different at the beginning of the study. However, if you have 100 participants in each group, the groups should be pretty similar before the start of the study. If they are very similar at the start, then, if they are even slightly different at the end, that difference could be due to the treatment. Similarly, in sports, if one team beats another in a seven game series that's more convincing evidence of the team's superiority than winning a single game.)
Two possible verdicts from statistical tests
1. statistically significant:you are sure beyond a reasonable doubt (your doubt is less than 5%) that the difference between your groups is too big to be due to chance alone.
So, if the difference between the treatment group and the no-treatment group is too big to be due to chance alone, then some of that difference is probably due to treatment. In other words, the treatment probably had an effect.
2. not statistically significant:
you are not sure, beyond a reasonable doubt, that the difference between the groups is due to anything more than just chance.
So, you can't conclude anything. The results are inconclusive.
Time to take a time out to test your understanding of statistical significance.
Conclusions about the Experiment
The goal of the simple experiment:
is to find the causes of behavior. That is, the goal is to find rules that will allow us to understand and control behavior.
Why the simple experiment can accomplish that goal:
it uses random assignment. Random assignment accomplishes two things:
- It makes the groups similar before the start of the experiment
- It allows us to use statistics to determine how unlikely it is that the differences between the groups at the end of the experiment are due to chance alone. If it is extremely unlikely that the results are due to chance alone, we conclude that the treatment was at least partly responsible for the differences between the two groups.
Implications of the fact that the experiment depends on random assignment:
- Much research, even that done in labs, is not experimental research because it does not involve randomly assigning people to groups.
- Some field research is experimental because it involves randomly assigning people to groups.
- We have to do statistical tests to see if our treatment had an effect. We can't just say the treatment had an effect because the treatment group scored higher than the control group. We have to do statistics to see whether this difference was (probably) too big to be just a coincidence.
Take another ungradedquiz.
Review of the Simple Experiment
- Explain why there are certain research questions that cannot be answered by
using an experiment.
- Explain why the case study illustrates the value of having a control
group.
- Explain why the null hypothesis can't be proven right.
- Explain why researchers use double-blind techniques.
- Explain why only experiments can answer causal questions.
- Explain why a good hypothesis is one that could be disproven.
- Explain the difference between an independent variable and a dependent
variable.
- State a rule for knowing whether a study that uses two groups is
an experiment. State two implications of that rule.
- Explain why the following statement is false: "To find out if the treatment
had an effect, simply see if the treatment group scored differently than the
no-treatment group."
- Explain what it means if the results are statistically significant.
- Explain what you can conclude if the results are not statistically
significant.
- Explain why the simple experiment is better than the before-after study.
- Explain the difference between random assignment and random sampling.
- Explain why you can't scientifically study the effects of race, gender, or
personality.
- Take this quiz until you can easily get 100% on it.
- One last quiz on experiments