The American Association for the Advancement of Science, Capacity Center is conducting a forum in which representatives from key stakeholder groups address the challenges involved in conceptualizing, translating, and implementing traditional and innovative evaluation approaches, strategies, and methods to assess efforts which hold the promise of transforming undergraduate STEM education. The workshop takes initial steps to foster a multi-disciplinary community that engages in cross-disciplinary dialogue and work to develop appropriate theoretically and intellectually rigorous evaluations of undergraduate STEM education. Intellectual Merit: The workshop brings members of the evaluation community and undergraduate STEM disciplinary communities together to 1) examine and improve the conceptualization and implementation of rigorous evaluation of undergraduate STEM education innovation; 2) share lessons learned and best evaluation practices; and 3) investigate and stimulate the development and/or use of innovative or traditional evaluation approaches, strategies, methods, and tools. Given that a major goal of the Division of Undergraduate Education is to transform undergraduate education, the workshop is a forum for stakeholders to examine promising evaluation approaches, strategies, methods, and tools to assess progress toward the achievement of this goal. Broader Impact: Participants at the workshop have roots in a variety of institutions. They represent a cross section of STEM disciplinary faculty who are engaged in the development and/or evaluation of undergraduate STEM educational efforts and professional evaluators who are engaged in the evaluation of undergraduate STEM education projects. Participants also represent key designers/implementers and consumers of evaluation of undergraduate STEM educational innovations.
In the world of Science, Technology, Engineering and Mathematics (STEM) education, there are two broad categories of participants -- sponsors and performers. When it comes to assessing how project investments by the former have made a difference due to the activities of the latter, the plot thickens. Too often sponsor's expectations are not fully met, in part because performers either lack the expertise, the budget, or the interest in evaluation as well as dissemination with a broader community. Alternatively, the sponsors have sometimes not made clear what they consider the important "take-aways" or "lessons-learned" from a project that might be undertaken or what form of evidence they need. Both sponsors and performers seek better outcomes for undergraduate students and the organizations that prepare them for the next phases of their studies and their carrers. Yet this failure to communicate, or at least frustration, is due to variations within the two categories of stakeholders that represent different cultures, values, traditions and practices. The challenge is to listen to those from different acadmic cultures and meld their tools and concerns into a community that can both expand its repertoire and begin to converge on what makes sense along the investment-activity continuum dedicated to undergraduate STEM education. In 2010, NSF awarded the Center for Advancing Science & Engineering Capacity, American Association for the Advancement of Science (AAAS) a grant to support a workshop for evaluating undergraduate STEM education efforts, such as those funded by the Division of Undergraduate Education (DUE) programs in the Directorate for Education and Human Resources (EHR). The AAAS Capacity Center conducted a forum in which representatives from key stakeholder groups addressed the challenges involved in conceptualizing, translating, and implementing traditional and innovative evaluation approaches, strategies, and methods to assess efforts which hold the promise of transforming undergraduate STEM education. The workshop too the initial steps to foster a multi-disciplinary community that engages in cross-disciplinary dialogue and work to develop appropriate theoretically and intellectually rigorous evaluations of undergraduate STEM education. The workshop, Building Bridges across Disciplines: Evaluation in Science, Technology, Engineering and Mathematics (STEM) Undergraduate Education, was held on June 9, 2011, at AAAS Headquarters in Washington, DC. The workshop brought together three stakeholder groups: those with expertise in a STEM discipline who specialize in postsecondary education and educational research and evaluation, professionals trained in or conducting project or program evaluation as applied to a STEM discipline, and NSF program officers who are responsible for monitoring and accounting for progress in DUE programs. One result of this day-long conversation was a renewed concern for "nuanced measurement"--indicators that capture processes and outcomes of teaching, learning, engagement, and student "self-efficacy," among other objectives, that various NSF undergraduate education programs support. Three DUE programs with the overarching goal of excellent in undergraduate STEM education--STEP, TUES, and ATE--target the development of curriculum materials, innovative pedagogy, and adaptations of other classroom/laboratory practices. The attendees suggested that instead of focusing on separate programs, it would be useful to identify common processes that are encouraged as a means to achieving various outcomes. A survey was also sent to leading scientists, researchers, and educations, who have received funding for NSF undergraduate education programs for more than 5 years, in order to demonstrate how NSF undergraduate STEM education funding impacted teaching and learning. Questions included: In what year did you first receive NSF undergraduate STEM eduation funding? Briefly describe the undergraduate STEM education resources, methods or strategies that you developed as part of your NSF grant. If applicable, please provide data that indicates that your undergraduate STEM education resources, methods or strategies improved student learning? teaching practices? Have undergraduate STEM education resources, methods or strategies been used by other institutions? If yes, please explain, including number of institutions. A report on the AAAS-NSF Workshop held on June 9, 2011 was submitted November 27, 2011. The report, Building Bridges across Disciplines: Evaluation in Science, Technology, Engineering and Mathematics (STEM) Undergraduate Education, was written by Daryl E. Chubin, Barbara Bogue, Joanne Cohoon, Anthony L. DePass, James Dorsey, and Alan Peterfreund.
