This award funds two conferences in the fall of 2012 that bring together leading scholars to discuss issues that are important for the future of survey research methods in the social, behavioral, and economic sciences. High quality survey research is critical for these sciences. However, the cost of survey methods continues to increase. New insights into best practices on optimal measurement suggest new procedures might be a path to maximizing response rates and data accuracy.
The conferences will address two separate issues: survey methods and data linking and dissemination. The conferences will bring together leading experts to help develop a research agenda in survey methods. The results of the conferences will include a written report and a website that will document the conference presentations via recordings and visuals.
Because many government agencies and private firms also use survey methods to gather information, the broader impacts of better surveys will be substantial.
Outlining the opportunities and challenges facing survey research entails illuminating current knowledge about how best to maximize opportunities, cope with challenges, and maintain data quality while incurring affordable costs. The outputs of this project provide guidance for researchers, funding agencies, commercial companies, and other organizations engaged in survey research as to how best use their resources to conduct and support research through survey data collection. Specifically, the materials produced by this project address the following questions: 1. What are the challenges facing survey-based data collection today (e.g., falling participation rates, rising costs, or coverage of frames)? 2. What innovations in survey methodology have taken place or are on the horizon? 3. How should researchers and organizations think about survey data in the context of the explosion of new digital sources of data? Are there opportunities for blending data or mixed source methods that integrate existing administrative, commercial, or social media data with existing surveys to answer social science questions? 4. Given current challenges faced by survey research as well as the potential opportunities presented by new approaches to survey research, what types of questions will we be able to address with surveys in the future? The materials produced by this project address these four questions—which are about the current and future status of survey research in general (as opposed to exclosively about NSF funded surveys)—by drawing on the results of presentations commissioned from leading experts at two conferences held at NSF in fall of 2012. The two conferences brought together leading scholarly experts on topics that fit into four broad areas. First, discussion on challenges faced in conventional survey research covered a broad landscape, including key topics such as: probability versus non-probability sampling methods; multi-mode survey techniques; optimizing response rates and how nonresponse affects survey accuracy; use of incentives in survey collection; survey design, visual displays and cognitive evaluation of survey instruments; proxy reporting; interviewing techniques and challenges; confidentiality, respondent attrition and data attrition; and computation of survey weights. The second category of exploration focused on opportunities to expand data collection, including: paradata; the use of leave-behind measurement supplements and biomarkers; and specialized tools for measuring past events. Third, several methods of linking survey data with external sources was discussed, specifically: improving government, academic and industry data-sharing opportunities; linking survey data to official government records or with the Catalist Commercial Database; linking knowledge networks web panel data with external data; and the use of election administration data with other datasets. Lastly, discussion turned to improving research transparency and data dissemination, with a focus on: data curation; evaluating the usability of survey project websites; and the broader topic of the credibility of survey-based social science. Throughout the proceedings, conference participants explored steps that can be taken to enhance the value of survey methodology to a wide range of users, in academia, government, and the private sector. The conferences yielded several useful outcomes, including: (1) insights about how surveys should be done today to maximize data quality (thereby specifying how major infrastructure surveys should be designed and carried out), (2) important challenges facing the methodology, (3) best practices in data dissemination and data collection procedure documentation, (4) approaches that would be most desirable for large-scale infrastructure surveys to implement, and (5) research questions that merit future investigation. Four significant material outputs have resulted from this project: (1) the conferences, which brought together large groups of top scholars to share information and consolidate up-to-date insights, (2) a book to be published commercially, (3) a journal review article, and (4) a website that hosts the content produced for the conferences and provides an easily accessible respository for the government, industry, and academic survey research community (https://iriss.stanford.edu/content/future-survey-research-nsf). As of Feb 2, 2015, 2,495 unique visitors have accessed materials online at the website, and a total of 4,641 pageviews have occurred.
