POS 6757: SURVEY RESEARCH
Spring, 2011

Dr. Kenneth Wald
Email
Home Page
Office: 303 Anderson Matherly 12
Phone: 273-2391 or 392-0262 (Dept. office) Thursday, periods 8-10
Email: kenwald@ufl.edu 3:00-6:00
Office Hours:  Tue  1:00-5:00, Weds. 1:00-3:00 or by appointment

Background: The essence of survey research consists of selecting a sample of respondents, asking them questions, and analyzing their answers. Since its development in the 1930s and 1940s, this seemingly simple technique has become a major tool of investigation for academic researchers, journalists, social analysts, government agencies, political practitioners, market researchers, and others. The course is intended to provide students with a solid grounding in the design  of surveys, paying particularly close attention to the potential pitfalls inherent in this method of data collection. During the semester, we will address the question of when and why to undertake this particular form of research, explore the issues that arise in the administration and analysis of surveys, and consider how best to communicate survey data and issues. The principal focus of the seminar is on recognizing and dealing with potential threats to the validity of surveys that arise from what is called non-sampling error. Nonsampling error has been defined in many ways but never so memorably as when Robert Chambers described survey data as
. . . aggregates of what has emerged from fallible programming of fallible punching of fallible coding of responses which are what investigators wrote down as their interpretation of their instructions as to how they were to write down what they believed respondents said to them, which was only what respondents were prepared to say to them in reply to the investigator's rendering of their understanding of a question and the respondent's understanding of the way they asked it; always assuming that the interview took place at all and that the answers were not more congenially compiled under a tree or in a teashop or bar without the tiresome complication of a respondent. (R. Chambers, Rural Development: Putting the Last First [Essex, UK: Longman], 1983, pp. 51-2)

Beyond the customary topics, the seminar will also consider the use of new survey modes associated with advanced computer technology and surveys as a global phenomenon. At the end of the semester, you should be a more sophisticated consumer of surveys and also have the capacity to contribute intelligently to survey design and interpretation.

Reading: We will use the following paperback book as the core text for the seminar:

Groves, Robert et al. 2009. Survey Methodology. 2nd edition. New York: Wiley-Interscience

In addition, I've recommended a supplemental text that gives a more "how-to" feel:

Salant, Priscilla and Don A. Dillman.  1994. How to Conduct Your own Survey. New York: John Wiley, 1994

Except for two excerpts indicated below, this book is not required but I'll indicate readings keyed to the course schedule. Because the book is rather dated, let me suggest you look at this brief article as a supplement on web-based surveys. A recent report by a blue-ribbon taskforce has produced a comprehensive report on all the issues raised by online surveys

For those of you who anticipate taking a comprehensive examination over this material, I also recommend reading The Psychology of Survey Response by Roger Tourangeau, Lance J. Rips, and Kenneth A. Rasinski (New York: Cambridge University Press, 2000).

The texts will be supplemented by a substantial number of journal articles that will be available online and some supplementary material that will be distributed.

If you use an on-campus computer that's connected to the UF network, clicking on the links will bring up the article text or a gateway to it immediately. If you login from off campus via a third-party server such as AOL or Cox express, you'll need to configure your browser for proxy access or use the VPN program. Click here for directions about the proxy server (which only requires a simple login procedure, and here for information about VPN access which requires downloading and configuring some software. Although VPN is more difficult to set up, it makes using the site much easier. The proxy serverl requires that you use the library's website to locate the individual articles while VPN works directly from this web page. Whichever option you chooseI strongly recommend that you download all the readings from the web ASAP and store them on disk or your hard drive.

Responsibilities : POS 6757 is a graduate seminar and its success depends on you keeping up with the readings, attending class faithfully, participating actively in class activities, and contributing to discussion. Your final grade will reflect this goal by weighing several components:
  • Participation (5%) - You will be responsible for all general readings indicated on the syllabus. I will factor in both the frequency and quality of your contributions to class discussions generally. Absences (other than those due to illness or compelling circumstances) will count heavily against the participation grade. Evidence of timely reading of the assigned sources will weigh heavily in this component.
  • Presentation (20%) - Each student will be assigned  responsibility for presenting and leading discussion about the readings (or a portion thereof) for the week. As part of this assignment, you will need to be able to summarize the readings clearly and comprehensively and link them to the major topical concerns of the class. (You should not spend so much time on this that other students feel they can avoid reading the articles on their own!)  Beyond summarizing the readings, the principal task of the presenter is to encourage and facilitate discussion and debate. You can and should expect to draw on the reaction papers of your fellow students.
  • Reaction Papers (20% ) - Every third week, you will prepare a reaction paper in response to the assigned readings. I'm going to try a new wrinkle on this assignment for the semester. On the weeks you're assigned a paper, select one of the readings (not from the textbook) and think in a disciplined way about its relevance to your own potential research. The paper should be equally divided into two parts.  In the first half, describe and summarize the arguments of at least one of the articles:  What are the major themes and conclusions? Where is the author coming from?  The second part of your paper should describe how the article speaks to your own research interests. How might you use it in your research agenda? Papers should be 2-3 pages and may be left for me by 3:00 on Wednesday in the Political Science office or sent by email by 3:00 on Wednesday. You are also responsible for getting copies to the student responsible for the week's presentation. Late or missing papers will be factored into your grade for this component.
  • Mid-term Examination (30%) - The midterm, a take-home, will consist of a short number of essay questions that ask you to integrate the reading material. It will be distributed on February 17th and is due on February 24th. Submit this in hard copy and electronically via Turnitin.
  • Article Review (25%) - In consultation with the instructor, you will select one  substantial article in your field that relies heavily on survey data and addresses one of the methodological concerns raised in the course. (This may not be an assigned reading.) You are to write a methodological review of the article that describes and summarizes the contents and evaluates the use of survey data in the context of the issues and themes in the course readings.  You will need to clear the choice with me. The review is due on April 26th (Tuesday).  Submit this in hard copy and in machine-readable form via Turnitin.
As in any graduate seminar unexcused absences, missed deadlines and lack of participation will count against you.
Outline:
Note: The required book chapters will be available on E-reserve unless otherwise indicated. POQ refers to Public Opinion Quarterly, AJPS to American Journal of Political Science, JOP to Journal of Politics. The Salant & Dillman readings indicated with an asterisk are optional.
1. Orientation (1/6)

2. The Concept of  Survey Error (1/13)

Survey Methodology, chaps. 1-2

A. Error Sources & Challenges
Henry Assael and John Keon, "Nonsampling vs. Sampling Errors in Survey Research," Journal of Marketing 46 (1982), 114-123.
Mitchell Seligson, "Improving the Quality of Survey Research in Developing Nations," PS: Political Science & Policy 38 (2005), 51-56.
*Salant & Dillman, Designing, chs. 1-2

B. What do Surveys Measure?
Phillip Converse, "Nature of Belief Systems in Mass Publics," Ideology and Discontent, 227-245
_____, "Attitudes and Non-Attitudes: Continuation of a Dialogue," Quantitative Analysis of Social Problems," 168-180
John Sullivan et al., "Ideological Constraint in the Mass Public: A Methodological Critique and Some New Findings," AJPS 22 (1978), 233-249.
John Zaller and Stanley Feldman, "A Simple Theory of the Survey Response," AJPS 36 (1992), 579-616

3. Design Effects (1/20) [Disc: Dahan, Knowlton] CRP=A

Survey Methodology, chap. 5
Salant & Dillman, Designing, ch. 7, pp. 101-127; ch. 8, 137-148 (Note: This is a required reading for all students.)
*Salant & Dillman, Designing, ch. 4

Vidal Díaz de Rada, "Mail Surveys Using Dillman's TDM in Spain," Int. Journal of Public Opinion Research 13 (2001), 159-172.
Nadine Koch & Jolly Emrey, "The Internet and Opinion Measurement: Surveying Marginalized Populations," Social Science Q. 82 (2001), 131-8.
John N. Bassili, "Response Latency vs. Certainty as Indexes of the Strength of Voting Intentions in CATI," POQ 57 (1993), 54-61
Paul M. Sniderman and Douglas B. Grob, "Innovation in Experimental Design in Attitude Surveys," Annual Review of Sociology 22 (1996), 377-399
Jason Barabas & Jennifer Jerit, "Are Survey Experiments Externally Valid?," APSR 104 (2010), 226-242.
William S. Aquilino, "Interviewer Mode Effects in Surveys of Drug and Alcohol Use," POQ 58 (1994), 210-240
Paul C. Hewett et al., "Feasibility of Computer-Assisted Survey Interviewing in Africa: Experience from Rural Districts in Kenya." Social Science Computer Review 22 (2004): 319-334.

4. Questions & Answers in Survey Research (1/27 [topic A], 2/3 [topic B], 2/10 [topic C], 2/17 [topic D])

Survey Methodology, chaps. 7-8
*Salant & Dillman, Designing, chs. 33, 6-7

A. Question Design [Disc: Perdue ] CRP=B
Tom W. Smith, "That Which We Call Welfare by Any Other Name Would Smell Sweeter," POQ 51 (1987), 75-83
Raymond J. Adamek, "Public Opinion and Roe v. Wade : Measurement Difficulties," POQ 58 (1994), 409-418
Norman Bradburn and Carrie Miles, "Vague Quantifiers," POQ 43 (1979), 92-101
Jon Krosnick & M. Berent, "Comparisons of Party Identification & Policy Preferences: Impact of Survey Question Format," AJPS 37 (1993), 941-64
John A. McCarty & L. J. Shrum, "Measurement of Personal Values in Survey Research.: Alternative Rating Procedures," POQ  64 (2001), 271-98

B. Developing & Testing Survey Questions and Scales [Disc: Kanyilmaz  ] CRP=C
Howard Schuman, "Ordinary Questions and Policy Questions," POQ 50 (1986), 432-442 [You will have to scroll ahead a few pages.]
Jon Krosnick, "Respnse Strategies for Coping With Cognitive Demands of Attitude Measures," Applied Cognitive Psychology 5 (1991): 213-236
Diane Binson and Joseph Catania, "Respondents' Understanding of the Words Used in Sexual Behavior Questions," POQ 62 (1998), 190-208
Amy O'Donnell et al., "Using Focus Groups to Improve the Validity of Cross-National Survey Research," Qualitative Health Research 17 (2007): 971-981
Marcia Pearl and M. Paz Galupo, "Development and Validation of the Attitudes Toward Same-Sex Marriage Scale," Journal of Homosexuality 53 (2007): 117-134.

C. Cultural Influences [Disc: Baysal] CRP=A
Debra Javeline, "Response Effects in Polite Cultures," POQ 63 (1999), 1-28.
Judith L. Gibbons et al., "Researching Gender-Role Ideologies Internationally and Cross-Culturally," Psych. of Women Q 21 (1997), 151-170.

Adina Smith et al., "Development & Implementation of a Culturally Sensitive Cervical Health Survey: A Community-Based Participatory Approach." Women & Health 40 (2004): 67-86.

D. Sensitive Issues [Disc: Panikowski] CRP=B
Matthew Streb et al., "Social Desirability Effects and Support for a Female Presidential Candidate," POQ 72 (2008), 76-89
Jon Hurwitz and Mark Peffley, "Playing the Race Card in the Post Willie Horton Era," POQ 69 (2005), 99-112.
Jennifer Solomon et al., "Estimating Illegal Resource Use with Randomized Response Technique," Human Dimensions of Wildlife 12 (2007),  75-88
Brian Duff et al., "Good Excuses: Understanding Who Votes," POQ 71 (2007), 67-90

Take-home examination distributed in class on February 17th and is due by the start of class on February 24th.


5. Interviewer Effects (2/24) Take-home examination due in class. [Disc: Harrigan] CRP=C

Survey Methodology, chaps. 9
Michael Schober and Frederick Conrad, "Does Conversational Interviewing Reduce Survey Measurement Error?" POQ 61 (1997), 576-602
Darren W. Davis, "The Direction of Race of Interviewer Effects Among African-Americans," AJPS (1997) 41, 309-22
Steven Finkel et al., "Race-of-Interviewer Effects in a Pre-election Poll," POQ 55 (1991), 313-3
Emily Kane and Laura Macaulay, "Interviewer Gender and Gender Attitudes," POQ 57 (1993), 1-28
Kevin Hill & Dante Moreno, "Language as a Variable: English, Spanish, Ethnicity, and Political Opinion," Hispanic J of Behav Sci 23 (2001) 208-28

6. Respondent Effects (3/3)  [Disc: Tanrioven]  CRP=A
John Katosh and Michael Traugott, "Consequences of Validated and Self-Reported Voting Measures," POQ 45 (1981), 519-35
Stanley Presser, "Is Inaccuracy on Factual Items Item-Specific or Respondent-Specific?" POQ 48 (1984), 344-55
Michael Traugott & John Tucker, "Strategies for Predicting Whether a Citizen Will Vote & Estimation of  Outcomes," POQ 48 (1984), 330-43
Adam Berinsky, "Two Faces of Public Opinion," AJPS 44 (1999), 1209-30.

Spring Break, March 5 - March 12

7. Sampling (3/17 [topic A], 3/24 [topic B], 3/31 [topic C])

Survey Methodology, chaps. 3-4, 6, 10
*Salant & Dillman, Designing, chs. 5

A. General Issues [Disc: Stringer]  CRP=B
Linchiat Chang and Jon A. Krosnick, "National Surveys via RDD Telephone Interviewing Versus the Internet." POQ 73 (2009): 641-678.
Scott Keeter et al., "Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Sample," POQ 70 (2006): 759-779
Paul Lavrakas et al., "The State of Surveying Cell Phone Numbers in the United States," POQ 71 (2007), 840-54.
Stephen Ansolabehere & Brian F. Schaffner. "Residential Mobility, Family Structure, and The Cell-Only Population." POQ 74 (2010): 244-59.
Adam S. Willcox et al., "Effects of  Financial Incentives on Response Rates and Item Nonresponse," Human Dimensions of Wildlife 15 (2010): 288-295

B. Sampling Difficult Populations/Situations I [Disc: Joon] CRP=C 
Margaret Newby et al., "Survey Research Among Women in Bangladesh," American Behavioral Scientist 42 (1998),  252-75
George Rothbart et al ., "On Finding and Interviewing the Needles in the Haystack," POQ 46 (1982), 408-421
Kevin W. Miller et al., "Street-Intercept Survey Method in an African-American Community," Am. J. of Public Health 87 (1997), 655-58.

C. Sampling Difficult Populations/Situations II [Disc: Dahan]  CRP=A
Bridget Lavelle et al.,  "Strategies for Surveys of American Indians," POQ 73 (2009): 385-403.
Bhutta, Christine Brickman, "Not by the Book: Facebook as a Sampling Frame" (December 6, 2010). Available at SSRN: http://ssrn.com/abstract=1721162
Edward J. Alessi & James I. Martin. "Conducting an Internet-Based Survey: Benefits, Pitfalls, and Lessons Learned." Social Work Research 34 2010): 122-128.

8.  Post-Collection Data Adjustments (4/7) [Disc: Forster ] CRP=B
Howard Acock, "Working with Missing Values," Journal of Marriage & the Family 67 (2005), 1012-28
Gary King et al., "Analyzing Incomplete Political Science Data: Multiple Imputation," American Political Science Rev 95 (2001), 49-69
Timothy Johnson et al. "Using Community-Level Correlates to Evaluate Nonresponse Effects in a Telephone Survey." POQ 70 (2006): 704-19.

9.  Legal/Ethical Issues in Survey Research (4/14) [Disc: Alenicheva]  CRP=C

Survey Methodology, chaps. 11
Mitchell Seligson, "Human Subjects Protection and Large-N Research" PS: Political Science & Politics 41 (2008): 477-482
Fred Morgan, "Judicial Standards for Survey Research," Journal of Marketing 54 (1990), 59-70
Lisa Fontes, "Ethics in Family Violence Research," Family Relations 47 (1998), 53-61.
Mark Orkin, "The Politics and Problematics of Survey Research," American Behavioral Scientist 42 (1998), 201-222
Richard Hamilton, "Work and Leisure: On Reporting Poll Results," POQ 55 (1991), 347-356
April 26 - Paper due by noon