Mon 20 - Fri 24 October 2014 Portland, Oregon, United States
Wed 22 Oct 2014 14:37 - 15:00 at Salon E - Domain Specific Languages Chair(s): Yannis Smaragdakis

Surveys can be viewed as programs, complete with logic, control flow, and bugs. Word choice or the order in which questions are asked can unintentionally bias responses. Vague, confusing, or intrusive questions can cause respondents to abandon a survey. Surveys can also have runtime errors: inattentive respondents can taint results. This effect is especially problematic when deploying surveys in uncontrolled settings, such as on the web or via crowdsourcing platforms. Because the results of surveys drive business decisions and inform scientific conclusions, it is crucial to make sure they are correct.

We present SurveyMan, a system for designing, deploying, and automatically debugging surveys. Survey authors write their surveys in a lightweight domain-specific language aimed at end users. SurveyMan statically analyzes the survey to provide feedback to survey authors before deployment. It then compiles the survey into JavaScript and deploys it either to the web or a crowdsourcing platform. SurveyMan’s dynamic analyses automatically find survey bugs, and control for the quality of responses. We evaluate SurveyMan’s algorithms analytically and empirically, demonstrating its effectiveness with case studies of social science surveys conducted via Amazon’s Mechanical Turk.

SurveyMan Slides (oopsla2014-tosch.pdf)4.96MiB

Wed 22 Oct
Times are displayed in time zone: (GMT-07:00) Tijuana, Baja California change

13:30 - 15:00: OOPSLA - Domain Specific Languages at Salon E
Chair(s): Yannis SmaragdakisUniversity of Athens
oopsla201413:30 - 13:52
Eric HolkIndiana University, Ryan R. NewtonIndiana University, Jeremy G. Siek, Andrew LumsdaineIndiana University
Link to publication
oopsla201413:52 - 14:15
Richard UhlerMIT-CSAIL, Nirav DaveSRI International
Link to publication
oopsla201414:15 - 14:37
Jeffrey BosboomMIT CSAIL, Sumanaruban RajaduraiNational University of Singapore, Weng-Fai WongNational University of Singapore, Saman AmarasingheMIT
Link to publication
oopsla201414:37 - 15:00
Emma ToschUniversity of Massachusetts, Amherst, Emery BergerUniversity of Massachusetts, Amherst
Link to publication File Attached