POS 6757: SURVEY RESEARCH
Spring, 2009
Section 9348

Dr. Kenneth Wald
Email
Home Page
Office: 303 Anderson Matherly 116
Phone: 273-2391 or 392-0262 (Dept. office) Thursday, periods 8-10
Email: kenwald@ufl.edu 3:00-6:00
Hours:  Tuesdays 1:00-5:00; Thursday 1:00-3:00; or by appointment

Background: The essence of survey research consists of selecting a sample of respondents, asking them questions, and analyzing their answers. Since its development in the 1930s and 1940s, this seemingly simple technique has become a major tool of investigation for academic researchers, journalists, social analysts, government agencies, political practitioners, market researchers, and others. The course is intended to provide students with a solid grounding in the design  of surveys, paying particularly close attention to the potential pitfalls inherent in this method of data collection. During the semester, we will address the question of when and why to undertake this particular form of research, explore the issues that arise in the administration and analysis of surveys, and consider how best to communicate survey data and issues. The principal focus of the seminar is on recognizing and dealing with potential threats to the validity of surveys that arise from what is called non-sampling error. Nonsampling error has been defined in many ways but never so memorably as when Robert Chambers described survey data as
. . . aggregates of what has emerged from fallible programming of fallible punching of fallible coding of responses which are what investigators wrote down as their interpretation of their instructions as to how they were to write down what they believed respondents said to them, which was only what respondents were prepared to say to them in reply to the investigator's rendering of their understanding of a question and the respondent's understanding of the way they asked it; always assuming that the interview took place at all and that the answers were not more congenially compiled under a tree or in a teashop or bar without the tiresome complication of a respondent. (R. Chambers, Rural Development: Putting the Last First [Essex, UK: Longman], 1983, pp. 51-2)

Beyond the customary topics, the seminar will also consider the use of new survey modes associated with advanced computer technology and surveys as a global phenomenon. At the end of the semester, you should be a more sophisticated consumer of surveys and also have the capacity to contribute intelligently to survey design and interpretation. Just like our text (p. 32), the seminar is intended "to provide the intellectual foundation needed to understand and address the problems that survey research projects pose."

Reading: We will use the following paperback book as the core text for the seminar:

Groves, Robert et al. 2004. Survey Methodology. New York: Wiley-Interscience

In addition, I've recommended a supplemental text that gives a more "how-to" feel:

Salant, Priscilla and Don A. Dillman.  1994. How to Conduct Your own Survey. New York: John Wiley, 1994

Except for two excerpts indicated below, this books is not required but I'll indicate readings keyed to the course schedule.

For those of you who anticipate taking a comprehensive examination over this material, I also recommend reading The Psychology of Survey Response by Roger Tourangeau, Lance J. Rips, and Kenneth A. Rasinski (New York: Cambridge University Press, 2000).

The texts will be supplemented by a substantial number of journal articles that will be available online and some supplementary material that will be distributed.

If you use an on-campus computer or login from off campus via the NERDC terminal server, clicking on the links will bring up the article text immediately. If you login from off campus via a third-party server such as AOL or Cox express, you'll need to configure your browser for proxy access or use the VPN program. Click here for directions about the proxy server (which only requires a simple login procedure, and here for information about VPN access which requires downloading and configuring some software. Although VPN is more difficult to set up, it makes using the site much easier. The proxy serverl requires that you use the library's website to locate the individual articles while VPN works directly from this web page. Whichever option you chooseI strongly recommend that you download all the readings from the web ASAP and store them on disk or your hard drive.

Responsibilities : POS 6757 is a graduate seminar and its success depends on you keeping up with the readings, attending class faithfully, participating actively in class activities, and contributing to discussion. Your final grade will reflect this goal by weighing several components:
  • Participation (5%) - You will be responsible for all general readings indicated on the syllabus. I will factor in both the frequency and quality of your contributions to class discussions generally. Absences (other than those due to illness or compelling circumstances) will count heavily against the participation grade. Evidence of timely reading of the assigned sources will weigh heavily in this component.
  • Presentation (15%) - Each student will be assigned  responsibility for presenting and leading discussion about the readings (or a portion thereof) for the week. As part of this assignment, you will need to be able to summarize the readings clearly and comprehensively and link them to the major topical concerns of the class. (You should not spend so much time on this that other students feel they can avoid reading the articles on their own!)  Beyond summarizing the readings, the principal task of the presenter is to encourage and facilitate discussion and debate. You can and should expect to draw on the critical reaction papers of your fellow students.
  • Critical Reaction Papers (20% ) - For most weeks, you will prepare a critical reaction paper in response to the assigned readings. Such papers are intended to encourage you to think in a disciplined way about the readings--what are the assumptions, meaning, and implications of the various articles under discussion. The paper should be equally divided into two parts.  In the first half, describe and summarize the arguments of at least one of the articles:  What are the major themes and conclusions? Where is the author coming from?  The second part of your paper should describe your critical reactions  Here each student has a lot of flexibility, but examples of questions the student might discuss include: What did the author get right (if anything), and what did she get wrong (if anything)?  Did the article speak to your own research interests?  What do you feel were the most thought-provoking points in the article?  Papers should be 2-3 pages and may be left for me by 3:00 on Tuesdays in the Political Science office or sent by email by 3:00 on Tuesday. You are also responsible for getting copies to the student responsible for the week's presentation. Late or missing papers will be factored into your grade for this component.
  • Mid-term Examination (30%) - The midterm, a take-home, will consist of a short number of essay questions that ask you to integrate the reading material. It will be distributed on February 19th and is due on February26th. Submit this in hard copy and electronically via Turnitin.
  • Article Review (30%) - In consultation with the instructor, you will select one  substantial article in your field that relies heavily on survey data and addresses one of the methodological concerns raised in the course. (This may not be an assigned reading.) You are to write a methodological review of the article that describes and summarizes the contents and evaluates the use of survey data in the context of the issues and themes in the course readings.  You will need to clear the choice with me. The review is due on April 28th (Monday).  Submit this in hard copy and in machine-readable form via Turnitin.
As in any graduate seminar unexcused absences, missed deadlines and lack of participation will count against you.
Outline:
Note: The required book chapters will be available on E-reserve unless otherwise indicated. POQ refers to Public Opinion Quarterly, AJPS to American Journal of Political Science, JOP to Journal of Politics. The Salant & Dillman readings indicated with an asterisk are optional.
1. Orientation (1/8)

2. The Concept of  Survey Error (1/15)

Survey Methodology, chaps. 1-2

A. Error Sources & Challenges
Henry Assael and John Keon, "Nonsampling vs. Sampling Errors in Survey Research," Journal of Marketing 46 (1982), 114-123.
Mitchell Seligson, "Improving the Quality of Survey Research in Developing Nations," PS: Political Science & Policy 38 (2005), 51-56.
*Salant & Dillman, Designing, chs. 1-2

B. What do Surveys Measure?
Phillip Converse, "Nature of Belief Systems in Mass Publics," Ideology and Discontent, 227-245
_____, "Attitudes and Non-Attitudes: Continuation of a Dialogue," Quantitative Analysis of Social Problems," 168-180
John Sullivan et al., "Ideological Constraint in the Mass Public: A Methodological Critique and Some New Findings," AJPS 22 (1978), 233-249.
John Zaller and Stanley Feldman, "A Simple Theory of the Survey Response," AJPS 36 (1992), 579-616

3. Mode Effects in Surveys (1/22) [Disc: Tecklenburg] CRP=A

Survey Methodology, chap. 5
Salant & Dillman, Designing, ch. 7, pp. 101-127; ch. 8, 137-148 (Note: This is a required reading for all students.)
*Salant & Dillman, Designing, ch. 4

Maria Krysan et al.,"Response Rates and Response Content in Mail vs. Face-to-Face Surveys," POQ 58 (1994), 381-399
Vidal Díaz de Rada, "Mail Surveys Using Dillman's TDM in Spain," Int. Journal of Public Opinion Research 13 (2001), 159-172.
Nadine Koch & Jolly Emrey, "The Internet and Opinion Measurement: Surveying Marginalized Populations," Social Science Q. 82 (2001), 131-8.
John N. Bassili, "Response Latency vs. Certainty as Indexes of the Strength of Voting Intentions in CATI," POQ 57 (1993), 54-61
Paul M. Sniderman and Douglas B. Grob, "Innovation in Experimental Design in Attitude Surveys," Annual Review of Sociology 22 (1996), 377-399
William S. Aquilino, "Interviewer Mode Effects in Surveys of Drug and Alcohol Use," POQ 58 (1994), 210-240

4. Questions & Answers in Survey Research (1/29, 2/5, 2/12, 2/19)

Survey Methodology, chaps. 7-8
*Salant & Dillman, Designing, chs. 33, 6-7

A. Question Design [Disc:  ] CRP=B
Tom W. Smith, "That Which We Call Welfare by Any Other Name Would Smell Sweeter," POQ 51 (1987), 75-83
Raymond J. Adamek, "Public Opinion and Roe v. Wade : Measurement Difficulties," POQ 58 (1994), 409-418
Norman Bradburn and Carrie Miles, "Vague Quantifiers," POQ 43 (1979), 92-101
Jon Krosnick & M. Berent, "Comparisons of Party Identification & Policy Preferences: Impact of Survey Question Format," AJPS 37 (1993), 941-64
John A. McCarty & L. J. Shrum, "Measurement of Personal Values in Survey Research.: Alternative Rating Procedures," POQ  64 (2001), 271-98

B. Developing & Testing Survey Questions and Scales [Disc: Knowles ] CRP=A
Howard Schuman, "Ordinary Questions and Policy Questions," POQ 50 (1986), 432-442 [You will have to scroll ahead a few pages.]
Jon Krosnick, "Respnse Strategies for Coping With Cognitive Demands of Attitude Measures," Applied Cognitive Psychology 5 (1991): 213-236
Diane Binson and Joseph Catania, "Respondents' Understanding of the Words Used in Sexual Behavior Questions," POQ 62 (1998), 190-208
Paul C. Beatty and Gordon B. Willis, "Research Synthesis: The Practice of Cognitive Interviewing," POQ 71 (2007): 287-311
Riley Dunlap et al., Measuring Endorsement of the New Ecological Paradigm: A Revised NEP Scale." J. of Social Issues 56 (2000), 425-442.

C. Cultural Influences [Disc: Weghorst] CRP=B
Debra Javeline, "Response Effects in Polite Cultures," POQ 63 (1999), 1-28.
Irvine Clarke III, "Extreme Response Style in Cross-Cultural Research," International Marketing Review 18 (2001), 301-324
Judith L. Gibbons et al., "Researching Gender-Role Ideologies Internationally and Cross-Culturally," Psych. of Women Q 21 (1997), 151-170.

D. Sensitive Issues [Disc: Kaya] CRP=A
Matthew Streb et al., "Social Desirability Effects and Support for a Female Presidential Candidate," POQ 72 (2008), 76-89
Jon Hurwitz and Mark Peffley, "Playing the Race Card in the Post Willie Horton Era," POQ 69 (2005), 99-112.
Jennifer Solomon et al., "Estimating Illegal Resource Use with Randomized Response Technique," Human Dimensions of Wildlife 12 (2007),  75-88
Brian Duff et al., "Good Excuses: Understanding Who Votes," POQ 71 (2007), 67-90
 
Take-home examination distributed in class on February 19th and is due by the start of class on February 26th.


5. Interviewer Effects (2/26) Take-home examination due in class. [Disc: Bolduc] CRP=B

Survey Methodology, chaps. 9
Michael Schober and Frederick Conrad, "Does Conversational Interviewing Reduce Survey Measurement Error?" POQ 61 (1997), 576-602
Darren W. Davis, "The Direction of Race of Interviewer Effects Among African-Americans," AJPS (1997) 41, 309-22
Steven Finkel et al., "Race-of-Interviewer Effects in a Pre-election Poll," POQ 55 (1991), 313-3
Emily Kane and Laura Macaulay, "Interviewer Gender and Gender Attitudes," POQ 57 (1993), 1-28
Kevin Hill & Dante Moreno, "Language as a Variable: English, Spanish, Ethnicity, and Political Opinion," Hispanic J of Behav Sci 23 (2001) 208-28

6. Respondent Effects (3/5)  [Disc: Hauck]  CRP=A

John Katosh and Michael Traugott, "Consequences of Validated and Self-Reported Voting Measures," POQ 45 (1981), 519-35
Stanley Presser, "Is Inaccuracy on Factual Items Item-Specific or Respondent-Specific?" POQ 48 (1984), 344-55
Michael Traugott & John Tucker, "Strategies for Predicting Whether a Citizen Will Vote & Estimation of  Outcomes," POQ 48 (1984), 330-43
Adam Berinsky, "Two Faces of Public Opinion," AJPS 44 (1999), 1209-30.
Kenneth O. Doyle, "Opinion Research in Indian Country." International Journal of Intercultural Relations 25 (2001) 511–30

Spring Break, March 10 - March 16

7. Sampling (3/19, 3/26, 4/2)

Survey Methodology, chaps. 3-4, 6
*Salant & Dillman, Designing, chs. 5

A. General Issues [Disc: Kane]  CRP=B
Cecelie Gaziano, "Comparative Analysis of Within-Household Respondent Selection Techniques," POQ  69 (2005), 124-57
Robert Groves, "Nonresponse Rates and Nonresponse Bias in Household Surveys," POQ 70 (2006): 646-675
Scott Keeter et al., "Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Sample," POQ 70 2006: 759-779
Paul Lavrakas et al., "The State of Surveying Cell Phone Numbers in the United States," POQ 71 (2007), 840-54.

B. Sampling Difficult Populations/Situations [Disc: Hunhi & Miller] CRP=A for Newby to Miller CRP=B for Lindsay to Burnham
Margaret Newby et al., "Survey Research Among Women in Bangladesh," American Behavioral Scientist 42 (1998),  252-75
George Rothbart et al ., "On Finding and Interviewing the Needles in the Haystack," POQ 46 (1982), 408-421
D. G. Pratt et al., "Local Community Attitudes to Wildlife Utilisation in . . . Mongolia," Biodiversity and Conservation 13 (2004), 591–613.
Kevin W. Miller et al., "Street-Intercept Survey Method in an African-American Community," Am. J. of Publich Health 87 (1997), 655-58.
Jo Lindsay, "Getting the Numbers: The Unacknowledged Work in Recruiting for Survey Research," Field Methods 17 (2005), 119-128.
Jon M. Denstadli, "Analyzing Air Travel: Comparison of Different Methods and Data Collection Procedures," J. of Travel Research 39 (2000), 4-10.
Stephen Sifaneck & A. Neaigus, "Ethnographic Accessing, Sampling, and Screening of Hidden Pops.," Addiction Res & Theory 7 (2001), 519-43
Gilbert Burnham et al., "Mortality after the 2003 Invasion of Iraq: A Cross-Sectional Cluster Sample Survey," Lancet  264 (2004), 1857-1864.

8.  Post-Collection Data Adjustments (4/9) [Disc: Lentijo] CRP=A

Survey Methodology, chaps. 10
Howard Acock, "Working with Missing Values," Journal of Marriage & the Family 67 (2005), 1012-28
Gary King et al., "Analyzing Incomplete Political Science Data: Multiple Imputation," American Political Science Rev 95 (2001), 49-69

9.  Legal/Ethical Issues in Survey Research (4/16) [Disc: Markowitz & Rogers]  CRP=B

Survey Methodology, chaps. 11
Mitchell Seligson, "Human Subjects Protection and Large-N Research" PS: Political Science & Politics 41 (2008): 477-482
Fred Morgan, "Judicial Standards for Survey Research," Journal of Marketing 54 (1990), 59-70
Lisa Fontes, "Ethics in Family Violence Research," Family Relations 47 (1998), 53-61.
Mark Orkin, "The Politics and Problematics of Survey Research," American Behavioral Scientist 42 (1998), 201-222
Richard Hamilton, "Work and Leisure: On Reporting Poll Results," POQ 55 (1991), 347-356
Eleanor Singer et al., "Confidentiality Assurances and Response," POQ 59 (1995), 66-7
April 28 - Paper due by noon