SPLASH 2014
Mon 20 - Fri 24 October 2014 Portland, Oregon, United States
Wed 22 Oct 2014 14:37 - 15:00 at Salon E - Domain Specific Languages Chair(s): Yannis Smaragdakis

Surveys can be viewed as programs, complete with logic, control flow, and bugs. Word choice or the order in which questions are asked can unintentionally bias responses. Vague, confusing, or intrusive questions can cause respondents to abandon a survey. Surveys can also have runtime errors: inattentive respondents can taint results. This effect is especially problematic when deploying surveys in uncontrolled settings, such as on the web or via crowdsourcing platforms. Because the results of surveys drive business decisions and inform scientific conclusions, it is crucial to make sure they are correct.

We present SurveyMan, a system for designing, deploying, and automatically debugging surveys. Survey authors write their surveys in a lightweight domain-specific language aimed at end users. SurveyMan statically analyzes the survey to provide feedback to survey authors before deployment. It then compiles the survey into JavaScript and deploys it either to the web or a crowdsourcing platform. SurveyMan’s dynamic analyses automatically find survey bugs, and control for the quality of responses. We evaluate SurveyMan’s algorithms analytically and empirically, demonstrating its effectiveness with case studies of social science surveys conducted via Amazon’s Mechanical Turk.

SurveyMan Slides (oopsla2014-tosch.pdf)4.96MiB

Wed 22 Oct

oopsla2014
13:30 - 15:00: OOPSLA - Domain Specific Languages at Salon E
Chair(s): Yannis SmaragdakisUniversity of Athens
oopsla2014141397740000013:30 - 13:52
Talk
Eric HolkIndiana University, Ryan R. NewtonIndiana University, Jeremy G. Siek, Andrew LumsdaineIndiana University
Link to publication
oopsla2014141397875000013:52 - 14:15
Talk
Richard UhlerMIT-CSAIL, Nirav DaveSRI International
Link to publication
oopsla2014141398010000014:15 - 14:37
Talk
Jeffrey BosboomMIT CSAIL, Sumanaruban RajaduraiNational University of Singapore, Weng-Fai WongNational University of Singapore, Saman AmarasingheMIT
Link to publication
oopsla2014141398145000014:37 - 15:00
Talk
Emma ToschUniversity of Massachusetts, Amherst, Emery BergerUniversity of Massachusetts, Amherst
Link to publication File Attached