The Advanced Technological Education (ATE) Evaluation Resource Center (ERC) focused on evaluating the work of the ATE program through an annual survey since 2000. The ERC now investigates and responds to the evaluation needs of the ATE projects by assisting current and prospective ATE grantees to develop high-quality evaluations that provide evidence to demonstrate the extent to which the goals of the projects and the ATE program are achieved. The Center provides ATE grantees and evaluators with support to design, conduct and report credible, useful improvement and accountability-oriented evaluations. The ERC continues to provide ATE program monitoring activities that are useful at the program and project level. The ERC assesses the evaluation needs and capacities of ATE grantees, collects and develops evaluation resources keyed to their needs and builds evaluation capacity. The Center also engages in and supports research on evaluation-related ATE issues and disseminates the findings and resources to the projects. Information about the Center activities is disseminated to increase capacity for educational evaluation more broadly. The activities of the Center are also evaluated.
EvaluATE is the evaluation resource center for NSF’s Advanced Technological Education program. EvaluATE’s mission is to promote the goals of the ATE program by partnering with ATE projects and centers to strengthen the program's evaluation knowledge base, expand the use of exemplary evaluation practices, and support the continuous improvement of technician education throughout the nation. Our goals are to (1) ensure that all ATE principal investigators (PIs) and evaluators know the essential elements of a credible and useful evaluation; (2) maintain a comprehensive collection of online resources for ATE evaluation; (3) strengthen and expand the network of ATE evaluation stakeholders; and (4) gather, synthesize, and disseminate data about ATE program activities to advance knowledge about the program and technician education in general. Intellectual Merit. The fundamental nature of EvaluATE’s work is that of evaluation capacity building, which is "work to continuously create and sustain overall organizational processes that make quality evaluation and its use routine" (Stockdill, Baizerman, & Compton, 2002). As such, it is geared toward supporting ATE grantees to use evaluation regularly to improve their work and demonstrate their projects’ impacts. From 2008-13, EvaluATE offered 25 webinars and 5 workshops, gave 36 conference presentations, and published 20 quarterly newsletters. These activities have directly served more than 1,000 people (not including website visitors). Over the five years of this grant, more than one-quarter of ATE principal investigators and more than one-half of ATE evaluators have actively participated with us by attending a live webinar or workshop. Additionally, EvaluATE has continued the survey of ATE grantees, which has been conducted annually since 2000, with an average response rate exceeding 90 percent. From 2008-13, we produced more than 20 reports and data snapshots based on results from this annual survey. We have produced an array of resources to support evaluation efforts in the ATE context, such as a checklist for evaluation planning for ATE proposals, a logic model template, an ATE evaluation primer, a guide to using Google Analytics for project evaluation, and numerous handouts that provide additional information and resources related to webinar and workshops topics. The results of the 2013 survey of EvaluATE’s constituency (which includes those targeted by EvaluATE and others who seek out our resources) conducted by EvaluATE’s external evaluator provided strong quantitative and qualitative evidence that EvaluATE is fulfilling its mission. For example, a majority of the survey respondents, which includes users and nonusers of EvaluATE’s resources, responded positively to the following points: EvaluATE has improved their understanding of evaluation in general (72%), how to incorporate evaluation in project planning (61%), and how to capture evidence of project impact (59%). They also said because of EvaluATE they were more effective in using evaluation results to improve projects (56%) and have taken steps to learn more about evaluation on their own (57%). Responses to these and other items on the survey indicate EvaluATE’s impact is greater for those who use our resources more frequently. Broader Impacts. Due to the very nature of the discipline, evaluation methods and results are inte nded to be applied within the diverse disciplinary contexts that funded projects are executed. ATE grantees focus on educating students and educators in the areas of manufacturing; agriculture and natural resources; energy production and use; biotechnology; chemical processes; optics; electronics and controls; marine technology; space technology; information and communications technologies; geospatial technologies; security, information assurance; and forensics—among others. Project-level evaluation in these contexts provides PIs with information at the grant-level that will help them improve project operations and results. At the program level, evaluation results help to advance understanding about what works and doesn’t work for improving the quality and quantity of technicians in the U.S. workforce. EvaluATE’s content is targeted to ATE grant recipients, but the content has broad relevance to the evaluation of educational programs in general. EvaluATE webinars attract a diverse audience, reaching far beyond the ATE community and drawing participants from federal agencies, independent consultancies, international consulting firms, nonprofits, universities, school districts, and state governments—as well as a few participants from outside the U.S.
