AIR will conduct a 3-year longitudinal study of the National Science Foundation?s Alliances for Graduate Education and the Professoriate (AGEP) program. The proposed project builds on the accomplishments of AIR AGEP studies and expands upon previous and the current AGEP focused national evaluation work. By renewing the existing grant, AGEP-LS will draw on extant data from the Survey of Earned Doctorates (SED) and the Survey of Doctoral Recipients (SDR) to track national enrollment and completion trends of underrepresented minorities in science, technology, engineering, and mathematics (STEM) fields as well as in the social, behavioral and economic sciences (SBE).
AIR will also deploy a software system, the AGEP-Tracking System and Registry (AGEP-TSR), to collect individual level data on AGEP students that will track their participation in AGEP supported activities at the alliance and campus levels. Through AGEP-TSR, the longitudinal study will be able to account for individual level outcomes and longer-term workforce outcomes. Through AGEP-TSR, AIR will systematically monitor performance of the participating AGEP Alliances over time and also be able to survey student participants at regular intervals. Finally, AIR will conduct intensive case studies over time with fixed cohorts of students, faculty, and AGEP program administrators within each of nine Alliances
Intellectual Merit. The proposed study would be the first to measure AGEP impact over time on a controlled population of students and faculty and to evaluate the effects of AGEP-supported institutional transformation activities over a measured time period. AGEP-LS also will be the first effort to evaluate the influence of the AGEP SBE program in these fields with minority students. As such, the study will advance what we know and understand about how AGEP affects the educational success of STEM and SBE doctoral students and the diversification of the professoriate.
Broader Impact. AGEP is explicitly designed to increase minority representation in STEM and SBE doctoral production and in the professoriate, but the field needs to know more about how program components work. Lessons learned about program efficacy over time through this study will be disseminated to the AGEP Alliances and to the broader higher education community. Through specific dissemination strategies, the study findings are intended to broaden the participation of underrepresented groups in graduate education and the professoriate in key fields.
American Institutes for Research (AIR) was initially awarded a grant to conduct a mixed-methods longitudinal evaluation of the National Science Foundation's (NSF's) Alliances for Graduate and the Professoriate (AGEP) grant program. Following the grant award, a revised scope of work was established to better address the new direction of the AGEP grant program and to meet the needs of the AGEP community. The revised scope of work drew on AIR's intellectual merit and strength as a large behavioral and social science research evaluation organization. The work transitioned the longitudinal study into specific research work to include specifically: National trend analyses of STEM graduate program enrollment, Ph.D. completion, time to completion, and postdoctoral employment status, disaggregated by institutional type (AGEP and non-AGEP), major STEM discipline, student race/ethnicity, and student gender Research briefs on issues affecting underrepresented groups' pathways to the STEM doctorate Data-driven AGEP learning community Expert conveninings. The key outcomes and broader impacts of these activities are: Trend analyses. AIR produced a final report that analyzed trends (1990-2010) in STEM graduate program enrollment, Ph.D. completion, time to completion and Ph.D. holders' postdoctoral career pathways. Differences across institutions and among students with different demographic characteristics and in different STEM fields were analyzed. Research briefs. AIR produced nine issue briefs on topics that included graduate student debt levels and institutional funding patterns, time to completion, the careers of STEM Ph.D. holders, the role of historically Black colleges and universities as institutional pathways to the STEM doctorate, and gender imbalances in STEM Ph.D. production. The briefs focused on differences among students with difference backgrounds and characteristics. The findings from the briefs were highlighted in the media and at conferences, public forums, and expert panel discussions. All of the briefs can be accessed here: www.broadeningstem.org Data-driven AGEP learning community. The learning community was facilitated by AIR with the support of Xcalibur, Inc. It included AGEP members who were interested in engaging in conversations about using data to inform practices aimed at building capacity to better support URM pathways to the STEM doctorate. A final report summarizing the activities and key observations and recommendations of the learning community was produced. The following factors were identified as key considerations in using data to inform and evaluate broadening participation in STEM at the doctoral level: Institutional capacity and challenges related to data reporting requirements Student record privacy Small sample sizes Access to timely data Expert convenings. AIR facilitated two expert convenings: an Evaluation Think Tank and a STEM Symposium. Evaluation Think Tank. The aim of this two-day think tank was to reconceptualize future evaluations to support the needs of federal programs that provide funding to build institutional capacity to change practices. The think tank convened a panel of experts in the field of study design and evaluation methodologies. The experts engaged in critical diaologue on issues to consider in developing and implementing evaluations of federal grant programs at teh program and project levels. The resulting white paper provided an overview of the meeting and a summary of the panel's key observations and recommendations on common design challenges and solutions, characteristics of capacity building programs to account for in designing evaluations, and strategies for linking program and project evaluations. The white paper was circulated among NSF program officers. STEM Symposium. In September 2013, AIR convened this two-day symposium, Using Research to Inform Policies and Practices in Science Education, to engage key stakeholders in a series of discussions on how social science research can inform broadening participation efforts in STEM. The key topics discussed included developing a science identity, stereotype threat, chilly climates, and science education in the current fiscal environment. The postsymposium report and all multimedia presentations, associated handouts, and audio recordings of the presentations are available at: www.broadeningstem.org/index.php/stem-symposium In sum, this grant resulted in a body of research and papers that provide in-depth data analyses on trends in STEM doctoral degree pathway participation and outcomes, and that highlight the myriad factors that interact in ways that differentially affect historically underrepresented groups of individuals. Building institutional capacity to support and sustain broadening participation efforts will require the commitment and engagement of multiple stakeholders. These stakeholders must have a firm understanding of the implicit and explicit factors that affect underrepresented individuals' experiences and abilities to persist in STEM for their efforts to prove fruitful. Our research shows that broadening participation efforts will require a context-based approach, but an approach informed by broader research and the lessons learned by the work of other institutions and departments. They must also include multiple supports and strategies that form an integrated, ecological approach to supporting students and changing departmental culture and practice. Only then will the potential for reversing chilly climates and biases against URMs and women of all races and ethnicities in academic STEM pathways present itself.
