This exploratory research study takes a first step toward characterizing measures of the broader impacts for the Research on Gender in Science and Engineering Program (GSE) at the National Science Foundation (NSF). A review of the GSE funded studies and products will be conducted to identify metrics and measures that are used in research on gender and as indicators of change. A well articulated logic model for evaluation and a set of benchmarks will result from interviews with principal investigators and the GSE program officer. Recommendations for measures of broader impact that may capture the intended interactions between research, dissemination, and extension service projects will be derived from the emergent program logic. Finally, a report of the broader impact measures and metrics identified for each of the three GSE Program components will be prepared, linking the evaluation logic model components to the strategic goals of NSF and to existing national data that frame the context for research on gender in science and engineering.
The intellectual merits of this proposal include the emphasis on metrics and measures that may be used to capture the dynamic interactions among contributors to the GSE learning community. Exploring various indicators and asking how to capture the nature of links between logic model components as well as the short and long term "broader impacts" of the GSE Program are also novel and on the cutting edge of measurement research. Through this study, implicit theories of action, underlying evaluation logic, and relevant metrics and measures will be identified. The proposed principal investigator (PI) and grantee organization bring strengths in drawing together knowledge of large scale data resources to triangulating quantitative and qualitative insights necessary for exploratory research. Some of this experience in measurement and evaluation has come from previous and successfully completed evaluations of NSF-EHR programs.
The broader impacts of this study will inform principal investigators by providing a focused review and annotated inventory of measures, protocols, and instruments that may be appropriate for their work. GSE researchers will benefit from the identification of knowledge gaps and testable assumptions that result from articulating the evaluation logic model and corresponding implicit theories of action. Finally, program officers, policy-makers, and the public will benefit by having an array of possible measures to use in evaluating the broader impacts of federally funded programs.
