This project seeks to develop methods to quantify and improve the generalizability of findings from STEM educational evaluation and research. A central problem in educational evaluation is how to determine whether the results of a well-designed experiment or quasi-experiment evaluating the effect of a STEM educational intervention (e.g., curricula, instructional strategies, professional development, or technologies) will apply to a different policy-relevant population.
This research will build on propensity score methods and a database of national covariates to create a statistical approach that uses study samples to estimate parameters of the distribution of treatment effects in an inference population. In short, this method will make it possible to use results from one study to make statistical claims in another population or place. This is something that cannot be done in most cases using current methods.
In addition to the mathematical derivation of new statistical methods, the project will develop a public-use database containing NCES Common Core of Data and census data linked to it as a resource for establishing covariate values for inference populations. The method will be tested and validated by analyzing secondary data from 20 field studies supported by NSF and the Institute of Education Sciences (IES) of the US Department of Education. The investigator will increase the field's capacity to use the new methods by conducting training workshops at four national educational conferences for other researchers.
