## Lecture TOPICS

1. Show how research findings have allowed us to be better aware of the limitations of the survey method and how to better conduct surveys (see Lecture 8.1). If you want to discuss specifics of how--or whether-- to label scale points, you may find this article useful.

2. Stress the importance of planning for conducting proper survey research (see Lecture 8.2). As part of your presentation, you might want to discuss how the Gallup organization conducts polls. (pdf file)

3. Go through this list of 20 questions journalists should ask about poll results.

4. Discuss the problems with self-report data. You can give students a brief and informative article on the pitfalls of self-report by copying pages 1 and 29 of the January, 1997 APA Monitor.

5. Discuss ways of dealing with response biases, such as by using the randomized response technique. Handout 8.1 shows the logic of the technique. Point out that, in research, investigators would use a different "innocuous" question. The only requirement for the innocuous question is that we know, based on previous research, what percentage of respondents will agree with the statement.

6. Review basic descriptive statistics. Either before or after your presentation, you might have students look at this on-line guide for helping nonstatisticians question statistical information.

7. Discuss more statistical issues, such as

1. the dangers of making too many comparisons,

2. the difference between significance testing and estimating effect size,

3. the implications of level of measurement on analysis and interpretation of analyses,

4. Common types of analyses such as Chi-square analysis, parameter estimation, and regression.
5. Using statistics to determine your sample size. To illustrate how this works, you could have students look at this simple explanation and table of sample size and sample accuracy.

## ACTIVE EXERCISES

8. Have the class critique surveys (Obtaining surveys is so easy--just don't throw away your junk mail for a few days or, if you're in a rush, you can probably get some surveys from your school's admissions department or your school's office of institutional research.

9. Laura Madson has devised a fun activity that helps students learn

• that the wording of surveys matters,
• that people tend to agree with statements (acquiescence response set, “yea-saying”),
• that surveys can be used as dependent measures in experiments, and
• the difference between experiments and surveys.

General strategy. Give one half the class  one  version of a short survey and the other half a different version. Other than variations in wording, the two surveys are the same. Administer the surveys and then have students compare responses to the two surveys.

## Ways to Implement the strategy

• You could have students use our computerized survey experiment. On arriving at that page, students randomly receive one of two versions of a six-item survey. Version 1 (Condition 1) leads students to (a) give more conservative responses because of the tendency to agree with items and because one of the questions is leading and (b) say that they study more (because students are free to interpret what studying " a lot" is). Version 2 differs from Version 1 in three main respects: (a) agreement with a Version 2 question (e.g., question 1, " Most women are better suited emotionally for politics than most men" is equivalent to disagreeing with the corresponding question on the Version 1 form:" Most men are better suited emotionally for politics than most women"), (b) a leading question ( "Given the failure of welfare in the United States, welfare programs should be eliminated.") is modified to "Welfare programs should be continued," and (c) students are told what "studying alot" is (more than 20 hours a week).

A data file containing each participant's condition number (Condition 1 or Condition 2) and each participant's responses to the six question (responses to the Condition 2 survey are reverse scored) is available from this link.

To understand the structure of the data file, suppose four participants all participated in the survey on January 18, 2005, between 8:28 and 8:35 in the morning, and all used the same computer (one with the IP address 80.204.176.13). The first participant was in Condition 1 and strongly disagreed with all six items. The second participant was also in Condition 1 but strongly agreed with all six items. The third participant was in Condition 2 and strongly disagreed with all six items (note that the computer will reverse score that participants' responses). The fourth participant received the second version of the survey and strongly agreed with all six items (again, note that the comptuer will reverse score responses from the second version of the survey). In that case, the data file will look like the following:

0/18/105, 8:33:1, 80.204.176.13, Condition 1, 1, 1, 1, 1, 1, 1 endline
0/18/105, 8:34:41, 80.204.176.13, Condition 1, 7, 7, 7, 7, 7, 7 endline
0/18/105, 8:28:41, 80.204.176.13, Condition 2, 7, 7, 7, 7, 7, 7 endline
0/18/105, 8:31:2, 80.204.176.13, Condition 2, 1, 1, 1, 1, 1, 1 endline

If you want to create a data file that contains only the data from your students, tell your students to put a certain number (e.g., 8) in the code number box. For example, if all your students put the number 8 in the code number box, their data will be stored in the file http://markwebtest.netfirms.com/data/randfakesurvey8.txt
If, on the other hand, they all put the number 66 in the box, their data will be stored in the file
http://markwebtest.netfirms.com/data/randfakesurvey66.txt

• Create or Have the Class Create the Surveys
• Use Professor Madson’s two, 8-item surveys . The questions for those surveys is in Table 1 (page 41) of her article.

• Give students a survey composed of bad questions (these bad questions  could come from Madson’s items). Then,  put students in small groups and have them use the nine editing tips on pages 197-199 of Research design explained to edit the bad survey questions.   Have one group propose  (by writing on the blackboard or on an overhead) their rewording of the first question. Once the class agrees with that change,  move to the next question. These revised questions will become the second survey.

• To administer the survey, you could

• Administer it yourself to a large class

or
• Have students administer it to a few friends

Madson, L. (2005). Demonstrating the importance of question wording on surveys. Teaching of Psychology, 32, 40-43.

10. Use Handout 5.5 in the Instructor's Manual to have students construct an attitude scale or a psychological test.

11. Conduct a survey in class. This exercise will help students see the distinction between a population and a sample, differences among sampling techniques, the problems of generalizing results from a sample to a population, the problem of self-report bias, and the importance of clearly worded questions. For a sample exercise, see

Carducci, B. J. (1996). Fighting shyness with shyness: An exercise in survey methodology and

self-awareness. Teaching of Psychology, 23, 241-243.

To get a software program that will allow you to administer the survey  from the web or from a desktop computer, download a trial version of InspireData.

12. Have students create their own surveys. For example, you might ask them to design a survey to find out who people's heroes were (and what qualities people thought a hero should be). Or, they could follow in Robert Sternberg's footsteps and see how people defined love or intelligence. Or, they could extend the research discussed in the chapter about the relationship between work and grade point average (GPA) for high school students by looking at the relationship between work and GPA for college students.

If you want students to do their survey online, you can have them set up free accounts at any of the following sites:

13. Work with the class to design a survey that might improve the quality of your course. An existing survey (The Student Assessment of Learning Gains Instrument) is available on-line. This survey can be modified, administered, and analyzed (means and standard deviations) on-line. In addition, you can download the raw data to do your own analyses. Last, but not least, it is free.

14. Have students report on a study using survey research. This link leads to article suggestions and handouts that help students understand those articles.
15. Introduce students to regression with this simple, fun, hands-on activity. For a briefer introduction to regression (and to emphasize the problems with outliers), guide students through this great applet.

16. Analyze real data from several surveys including the General Social Survey. The complete data set has over 35,000 cases and hundreds of variables. The variables represent nominal, ordinal, interval, and ratio scales. You can select years and/or variables and download them as text or SPSS files. Even more sources of survey data that can be analyzed are available from the following link.

Back to Chapter 8 Main Menu