POS 6757: SURVEY RESEARCH
Spring, 2013 (Section 8199)

Dr. Kenneth Wald
Email
Home Page
Office: 303 Anderson Anderson 32
Phone: 273-2391 or 392-0262 (Dept. office) Monday, periods 8-10
Email: kenwald@ufl.edu 3:00-6:00
Office Hours:  Thursdays, 1:30-4:30 or by appointment

Background: The essence of survey research consists of selecting a sample of respondents, asking them questions, and analyzing their answers. Since its development in the 1930s and 1940s, this seemingly simple technique has become a major tool of investigation for academic researchers, journalists, social analysts, government agencies, political practitioners, market researchers, and others. The course is intended to provide students with a solid grounding in the design  of surveys, paying particularly close attention to the potential pitfalls inherent in this method of data collection. During the semester, we will address the question of when and why to undertake this particular form of research, explore the issues that arise in the administration and analysis of surveys, and consider how best to communicate survey data and issues. The principal focus of the seminar is on recognizing and dealing with potential threats to the validity of surveys that arise from what is called non-sampling error. Nonsampling error has been defined in many ways but never so memorably as when Robert Chambers described survey data as
. . . aggregates of what has emerged from fallible programming of fallible punching of fallible coding of responses which are what investigators wrote down as their interpretation of their instructions as to how they were to write down what they believed respondents said to them, which was only what respondents were prepared to say to them in reply to the investigator's rendering of their understanding of a question and the respondent's understanding of the way they asked it; always assuming that the interview took place at all and that the answers were not more congenially compiled under a tree or in a teashop or bar without the tiresome complication of a respondent. (R. Chambers, Rural Development: Putting the Last First [Essex, UK: Longman], 1983, pp. 51-2)

Beyond the customary topics, the seminar will also consider the use of new survey modes associated with advanced computer technology and surveys as a global phenomenon. At the end of the semester, you should be a more sophisticated consumer of surveys and also have the capacity to contribute intelligently to survey design and interpretation.

Reading: We will use the following paperback book as the core text for the seminar:

Groves, Robert et al. 2009. Survey Methodology. 2nd edition. New York: Wiley-Interscience

In addition, I've recommended a supplemental text that gives a more "how-to" feel:

Salant, Priscilla and Don A. Dillman.  1994. How to Conduct Your own Survey. New York: John Wiley, 1994

Except for two excerpts indicated below, this book is not required but I'll indicate readings keyed to the course schedule. Because the book is rather dated, let me suggest you look at this brief article as a supplement on web-based surveys. A recent report by a blue-ribbon taskforce has produced a comprehensive report on all the issues raised by online surveys

For those of you who anticipate taking a comprehensive examination over this material, I also recommend reading The Psychology of Survey Response by Roger Tourangeau, Lance J. Rips, and Kenneth A. Rasinski (New York: Cambridge University Press, 2000).

The texts will be supplemented by journal articles that are available online and some supplementary material that will be distributed.

If you use an on-campus computer that's connected to the UF network, clicking on the links will bring up the article text or a gateway to it immediately. If you login from off campus via a third-party server such as AOL or Cox express, you'll need to configure your browser for proxy access or use the VPN program. Click here for directions about the proxy server (which only requires a simple login procedure, and here for information about VPN access which requires downloading and configuring some software. Although VPN is more difficult to set up, it makes using the site much easier. The proxy server requires that you use the library's website to locate the individual articles while VPN works directly from this web page. Whichever option you choose, I strongly recommend that you download all the readings from the web early and store them. As URLs tend to change haphazardly, please let me know ASAP if a link has failed so I can fix it.

Responsibilities : POS 6757 is a graduate seminar and its success depends on you keeping up with the readings, attending class faithfully, participating actively in class activities, and contributing to discussion. Your final grade will reflect this goal by weighing several components:
  • Participation (5%) - You will be responsible for all general readings indicated on the syllabus. I will factor in both the frequency and quality of your contributions to class discussions generally. Absences (other than those due to illness or compelling circumstances) will count heavily against the participation grade while timely reading of the assigned sources will add points to this component.
  • Presentation (20%) - Each student will be assigned  responsibility for presenting and leading discussion about the readings (or a portion thereof) for the week. As part of this assignment, you will need to be able to summarize the readings clearly and comprehensively and link them to the major topical concerns of the class. (You should not spend so much time on this that other students feel they can avoid reading the articles on their own!)  Beyond summarizing the readings, the principal task of the presenter is to encourage and facilitate discussion and debate. You can and should expect to draw on the reaction papers of your fellow students. Presenters must schedule an appointment with me no later than the Thursday preceding class. You will be expected to provide the class with some kind of visual presentation in Powerpoint or similar format.
  • Reaction Papers (15% ) - Every fourth week, you will prepare a reaction paper in response to the assigned readings. On the weeks you're assigned a paper, select one of the readings (not from the textbook) and think in a disciplined way about its relevance to your own potential research. The paper should be equally divided into two parts. In the first half, describe and summarize the arguments of at least one of the articles:  What are the major themes and conclusions? Where is the author coming from?  The second part of your paper should describe how the article speaks to your own research interests. How might you or your colleagues use it in your research agenda? Papers should be 2-3 pages and should be submitted electronically via Sakai by the Friday preceding class. You are also responsible for getting copies to the student(s) responsible for the week's presentation. Late or missing papers will be factored into your grade for this component.
  • Mid-term Examination (30%) - The midterm, a take-home, will consist of a short number of essay questions that ask you to integrate the reading material. It will be distributed on February 18th and is due on February 25th. Submit this in hard copy and electronically via Turnitin (available in Sakai).
  • Article Review (30%) - In consultation with the instructor, you will select one  substantial article in your field that relies heavily on survey data and addresses one of the methodological concerns raised in the course. (This may not be an assigned reading.) You are to write a methodological review of the article that describes and summarizes the contents and evaluates the use of survey data in the context of the issues and themes in the course readings.  You will need to clear the choice with me by no later than April 8th. The review is due on April 30th (Tuesday).  Submit this in machine-readable form via Sakai.
Outline:
Note: The required book chapters will be available on E-reserve unless otherwise indicated. POQ refers to Public Opinion Quarterly, AJPS to American Journal of Political Science, JOP to Journal of Politics. The Salant & Dillman readings indicated with an asterisk are optional.
1. Orientation (1/7)

2. The Concept of  Survey Error (1/14)

Survey Methodology, chaps. 1-2

A. Error Sources & Challenges
Henry Assael and John Keon, "Nonsampling vs. Sampling Errors in Survey Research," Journal of Marketing 46 (1982), 114-123.
Mitchell Seligson, "Improving the Quality of Survey Research in Developing Nations," PS: Political Science & Policy 38 (2005), 51-56.
*Salant & Dillman, Designing, chs. 1-2

B. What do Surveys Measure?
Phillip Converse, "Nature of Belief Systems in Mass Publics," Ideology and Discontent, 227-245
_____, "Attitudes and Non-Attitudes: Continuation of a Dialogue," Quantitative Analysis of Social Problems," 168-180
John Sullivan et al., "Ideological Constraint in the Mass Public: A Methodological Critique and Some New Findings," AJPS 22 (1978), 233-249.
John Zaller and Stanley Feldman, "A Simple Theory of the Survey Response," AJPS 36 (1992), 579-616

King Day - no class on January 21

3. Design Effects (1/28) [Disc: Olivieri, Sellers] CRP=A

Survey Methodology, chap. 5
Salant & Dillman, Designing, ch. 7, pp. 101-127; ch. 8, 137-148 (Note: This is a required reading for all students.)
*Salant & Dillman, Designing, ch. 4

Robert M. Groves. "Three Eras of Survey Research" POQ 75 (2011), 861-871.
Tom W. Smith.  2012. "Survey-Research Paradigms Old and New." International Journal of Public Opinion Research. Forthcoming.
Vidal Díaz de Rada, "Mail Surveys Using Dillman's TDM in Spain," Int. Journal of Public Opinion Research 13 (2001), 159-172.
John N. Bassili, "Response Latency vs. Certainty as Indexes of the Strength of Voting Intentions in CATI," POQ 57 (1993), 54-61
Paul M. Sniderman and Douglas B. Grob, "Innovation in Experimental Design in Attitude Surveys," Annual Review of Sociology 22 (1996), 377-399
William S. Aquilino, "Interviewer Mode Effects in Surveys of Drug and Alcohol Use," POQ 58 (1994), 210-240
Paul C. Hewett et al., "Feasibility of Computer-Assisted Survey Interviewing in Africa: Experience from Rural Districts in Kenya." Social Science Computer Review 22 (2004): 319-334.

4. Questions & Answers in Survey Research (2/4 [topic A], 2/11 [topic B], 2/18 [topic C], 2/25 [topic D])

Survey Methodology, chaps. 7-8
*Salant & Dillman, Designing, chs. 33, 6-7

A. Question Design [Disc: Park] CRP=B
Tom W. Smith, "That Which We Call Welfare by Any Other Name Would Smell Sweeter," POQ 51 (1987), 75-83
Raymond J. Adamek, "Public Opinion and Roe v. Wade : Measurement Difficulties," POQ 58 (1994), 409-418
Norman Bradburn and Carrie Miles, "Vague Quantifiers," POQ 43 (1979), 92-101
Jon Krosnick & M. Berent, "Comparisons of Party Identification & Policy Preferences: Impact of Survey Question Format," AJPS 37 (1993), 941-64
John A. McCarty & L. J. Shrum, "Measurement of Personal Values in Survey Research.: Alternative Rating Procedures," POQ  64 (2001), 271-98

B. Developing & Testing Survey Questions and Scales [Disc: Alkazemi] CRP=C
Howard Schuman, "Ordinary Questions and Policy Questions," POQ 50 (1986), 432-442 [You will have to scroll ahead a few pages.]
Jon Krosnick, "Response Strategies for Coping With Cognitive Demands of Attitude Measures," Applied Cognitive Psychology 5 (1991): 213-236
Diane Binson and Joseph Catania, "Respondents' Understanding of the Words Used in Sexual Behavior Questions," POQ 62 (1998), 190-208
Amy O'Donnell et al., "Using Focus Groups to Improve the Validity of Cross-National Survey Research," Qualitative Health Research 17 (2007): 971-981
Marcia Pearl and M. Paz Galupo, "Development and Validation of the Attitudes Toward Same-Sex Marriage Scale," Journal of Homosexuality 53 (2007): 117-134.

C. Cultural Influences [Disc: Oztas] CRP=A
Debra Javeline, "Response Effects in Polite Cultures," POQ 63 (1999), 1-28.
Judith L. Gibbons et al., "Researching Gender-Role Ideologies Internationally and Cross-Culturally," Psych. of Women Q 21 (1997), 151-170.

Adina Smith et al., "Development & Implementation of a Culturally Sensitive Cervical Health Survey: A Community-Based Participatory Approach." Women & Health 40 (2004): 67-86.

D. Sensitive Issues [Disc: Mwaba] CRP=B
Matthew Streb et al., "Social Desirability Effects and Support for a Female Presidential Candidate," POQ 72 (2008), 76-89
Jon Hurwitz and Mark Peffley, "Playing the Race Card in the Post Willie Horton Era," POQ 69 (2005), 99-112.
Jennifer Solomon et al., "Estimating Illegal Resource Use with Randomized Response Technique," Human Dimensions of Wildlife 12 (2007),  75-88
Brian Duff et al., "Good Excuses: Understanding Who Votes," POQ 71 (2007), 67-90
Spring Break, no class on March 4

Take-home examination distributed in class on February 18th and is due (electronically) by the end of the day on February 25th.

5. Interviewer Effects (3/11)  [Disc: Asim] CRP=C

Survey Methodology, chaps. 9
Michael Schober and Frederick Conrad, "Does Conversational Interviewing Reduce Survey Measurement Error?" POQ 61 (1997), 576-602
Darren W. Davis, "The Direction of Race of Interviewer Effects Among African-Americans," AJPS (1997) 41, 309-22
Steven Finkel et al., "Race-of-Interviewer Effects in a Pre-election Poll," POQ 55 (1991), 313-3
Emily Kane and Laura Macaulay, "Interviewer Gender and Gender Attitudes," POQ 57 (1993), 1-28
Kevin Hill & Dante Moreno, "Language as a Variable: English, Spanish, Ethnicity, and Political Opinion," Hispanic J of Behav Sci 23 (2001) 208-28

6. Respondent Effects (3/18)  [Disc: Newberry]  CRP=A
John Katosh and Michael Traugott, "Consequences of Validated and Self-Reported Voting Measures," POQ 45 (1981), 519-35
Stanley Presser, "Is Inaccuracy on Factual Items Item-Specific or Respondent-Specific?" POQ 48 (1984), 344-55
Michael Traugott & John Tucker, "Strategies for Predicting Whether a Citizen Will Vote & Estimation of  Outcomes," POQ 48 (1984), 330-43
Adam Berinsky, "Two Faces of Public Opinion," AJPS 44 (1999), 1209-30.

7. Sampling (3/25 [topic A], 4/1 [topic B], 4/8 [topic C])

Survey Methodology, chaps. 3-4, 6, 10
*Salant & Dillman, Designing, chs. 5

A. General Issues [Disc: Apar, Tunalilar]  CRP=B
David S. Yeager et al. "Comparing the Accuracy of RDD Telephone Surveys &nd Internet Surveys Conducted with Probability & Non-Probability Samples." POQ 75 (2012):  709-747.
Linchiat Chang and Jon A. Krosnick, "National Surveys via RDD Telephone Interviewing Versus the Internet." POQ 73 (2009): 641-678.
Scott Keeter et al., "Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Sample," POQ 70 (2006): 759-779
Stephen Ansolabehere & Brian F. Schaffner. "Residential Mobility, Family Structure, and The Cell-Only Population." POQ 74 (2010): 244-59.
Robin Lee et al., "Given Increasing Bias in Random Digit Dial Sampling, Could Respondent-Driven Sampling Be a Practical Alternative?" Annals of Epidemiology 21 (2011): 272-279.
Adam S. Willcox et al., "Effects of  Financial Incentives on Response Rates and Item Nonresponse," Human Dimensions of Wildlife 15 (2010): 288-295

B. Sampling Difficult Populations/Situations I [Disc: Crandall] CRP=C 
Margaret Newby et al., "Survey Research Among Women in Bangladesh," American Behavioral Scientist 42 (1998),  252-75
George Rothbart et al ., "On Finding and Interviewing the Needles in the Haystack," POQ 46 (1982), 408-421
Kevin W. Miller et al., "Street-Intercept Survey Method in an African-American Community," Am. J. of Public Health 87 (1997), 655-58.
Alice Bloch. "Survey Research with Refugees: A Methodological Perspective." Policy Studies 25 (2004): 139-151.

C. Sampling Difficult Populations/Situations II [Disc: Ngo]  CRP=A
Bridget Lavelle et al.,  "Strategies for Surveys of American Indians," POQ 73 (2009): 385-403.
Christine Brickman Bhutta, "Not by the Book: Facebook as a Sampling Frame," Sociological Methods & Research 41 (2012), 57-88.
Edward J. Alessi & James I. Martin. "Conducting an Internet-Based Survey: Benefits, Pitfalls, and Lessons Learned." Social Work Research 34 2010): 122-128.
Nadine Koch & Jolly Emrey, "The Internet and Opinion Measurement: Surveying Marginalized Populations," Social Science Q. 82 (2001), 131-8.

8.  Post-Collection Data Adjustments (4/15) [Disc: Madley] CRP=B
Howard Acock, "Working with Missing Values," Journal of Marriage & the Family 67 (2005), 1012-28
Gary King et al., "Analyzing Incomplete Political Science Data: Multiple Imputation," American Political Science Rev 95 (2001), 49-69
Timothy Johnson et al. "Using Community-Level Correlates to Evaluate Nonresponse Effects in a Telephone Survey." POQ 70 (2006): 704-19.

9.  Legal/Ethical Issues in Survey Research (4/22) [Disc: TBA]  CRP=C

Survey Methodology, chaps. 11
Mitchell Seligson, "Human Subjects Protection and Large-N Research" PS: Political Science & Politics 41 (2008): 477-482
Fred Morgan, "Judicial Standards for Survey Research," Journal of Marketing 54 (1990), 59-70
Lisa Fontes, "Ethics in Family Violence Research," Family Relations 47 (1998), 53-61.
Mark Orkin, "The Politics and Problematics of Survey Research," American Behavioral Scientist 42 (1998), 201-222
Richard Hamilton, "Work and Leisure: On Reporting Poll Results," POQ 55 (1991), 347-356
April 30 - Paper due electronically by 4:00pm


For Powerpoint presentations, click here.