Problems with missing data led Groves, Dillman, Eltinge, and Little (2002, p. xiii) to argue that """"""""[one] of the important scientific challenges facing survey methodology at the beginning of this century is determining the circumstances under which non-response damages inference to the target population. A second challenge is the identification of methods to alter the estimation process in the face of non-response to improve the quality of the sample statistics."""""""" This proposal responds to these dual challenges. The proposal outlines a research agenda seeking to integrate the extensive empirical research from the survey research literature on the causes of survey non-response (e.g., Groves 2006) with the technical econometric literature on sample selection (e.g., Heckman 1976). The marriage of these literatures will yield two types of benefits. First, the econometric approach to missing data relies on assumptions that are rarely met in practice (Bushway, Johnson, and Slocum 2007, Little 1985). This has limited the applicability of the approach. The proposal describes how proactive survey design can be used to meet the first assumption and how these same survey design features can be used to test the second assumption. Preliminary research shows the feasibility and practicality of the approach. The second type of fruit borne by the proposal regards the design of experiments and surveys. The traditional approach to experimental design chooses the number of subjects subject to a budget constraint and assigns half of subjects to treatment and half to control. When complete follow-up data are available, such a design minimizes the mean-squared-error (MSE) of treatment effect estimates (Bloom 1995). However, when complete follow-up data are not available, this result no longer holds. Current prescriptions for how to handle missing data involve rules-of-thumb, such as minimizing aggregate non-response rates. Preliminary results show that this rule of thumb can lead to worse precision of estimated treatment effects. The proposal describes methods for maximizing the precision of estimates using a combination of experimental and survey design. Missing data compromises the quality of sample statistics in nearly all research involving human subjects. The proposal shows how missing data problems can be corrected using statistical techniques together with modifications of standard survey protocols.
