The researchers in this project will use large longitudinal datasets that include rich arrays of descriptive variables as well as measures of science and mathematics achievement to identify promising covariates that may eliminate or greatly reduce selection bias in nonrandomized estimates of program effects in research and evaluations of interventions or programs for K-12 students. Simulated nonrandomized intervention and comparison groups with selection bias will be created from a selection of data from five de-identified, publically available longitudinal datasets. The researchers will then create propensity scores for each simulation based on the examination of the magnitude of the multiple correlations between covariates and the outcome and use these scores to assess how much of the selection bias they might eliminate. The identification of covariates that sufficiently account for enough selection bias so that the remaining selection bias might be ignored will strengthen the use of quasi-experimental study design in STEM education.
Randomized control designs in STEM education are well known as being the strongest causal design with the ability to provide unbiased estimates of the effects of programs or interventions on intended outcomes. With an adequate sample size to minimize the likelihood of chance differences, random assignment can be expected to equate the experimental groups on all baseline characteristics that might influence the outcomes of interest. However, randomized experiments are not always possible in situations where the effects of STEM education programs are of interest. For many practical and occasionally, ethical reasons, nonrandomized comparison group designs are frequently used in STEM education research and evaluation studies. The findings of this study will provide empirical evidence of what covariate data should be collected to reduce selection bias in quasi-experimental designs.
