Successful ATE projects and centers are expected to be sustained and have an impact. The goal of this project is to better understand the factors that help sustain successful projects and centers and how those factors can be used to inform ongoing and future projects and centers. The project builds on previous research in which 216 ATE projects and centers responded with degree of agreement to peer-generated statements on sustainability and impact using a Lickert scale. It was found that many components of projects and centers are sustained even if the structure of the project is not and many projects have an impact on their own institution, other institutions, faculty and students. The present project mines these and other data further to clarify what is meant by sustainability, to identify factors related to it and to create new ways of measuring sustainability. The impact data are analyzed to determine the factors that influence the impact that occurs. A review committee evaluates the draft and final reports.

Project Report

A concern of funding agencies, including the National Science Foundation (NSF), is the extent to which grant products and activities continue after Foundation funding ends. This persistence is called sustainability. The purpose of this targeted research grant was to investigate the sustainability of the Advanced Technological Education (ATE) program. What are the dimensions of sustainability? Was the work of ATE projects and centers being sustained? If so, what factors are important for successful sustainability? The first task was to develop a way to measure sustainability. I used a peer-generated process where those experienced with the program wrote statements about the impact and sustainability of their experiences with ATE. These statements became the items on a Likert-type survey that was administered to all ATE projects and centers during the spring of 2009. I received 212 responses, an 81% reply rate. The returned surveys were used to investigate the properties of the scale such as reliability and validity. I found the ATE Impact and Sustainability scales met or exceeded the standards for effective measuring instruments. I also found that the scales discriminated between groups. This supports the validity claims of the scale. In general, the results of this research support the notion that many of the activities and products of the ATE program are being sustained. Program changes have been institutionalized, and the collaborations the institutions have formed with industry partners have persisted. New professional development programs are in place, and faculty have improved their teaching methods and formed new collegial relationships. I carried out a secondary analysis of the survey data to illustrate a number of important measurement principles that should be followed by those doing research and evaluation on the ATE program. I examined the responses to the survey and found no evidence of nonresponse bias. That is, those that answered the survey were shown to have similar background characteristics as those who did not reply to the survey. I also conducted a replication study to determine if I repeated the study, would I get the same results? Several survey items were included in an annual survey of ATE PIs. I compared these responses with my own and found the results to be the same. Such findings increase the confidence one has regarding research findings. Another outcome of the study was the development of an ATE sustainability checklist for use by team leaders and project evaluators. The checklist was designed to help identify areas where sustainability has occurred, improve sustainability success, provide accountability evidence, and determine how a specific grant compares with the sustainability of other ATE projects and centers. The checklist is available free to all ATE grantees and those interested in doing research in the area. Several of the items on the survey were designed to measure the impact of an ATE experience. Many of these items were evaluative in nature so I selected a number of them to create an ATE Evaluation Survey. The survey is to help PIs and others meet NSF’s mandate that grants must be evaluated. The survey has been shown to be valid and reliable. Percentile ranks are provided so users of the survey can compare their scores with others who have used it. Many models provide suggestions on how to increase the sustainability of a new program. However, these models are based on personal experience, not on research results. To address this issue, I selected a model for ATE sustainability and carried out an exploratory test of its predictions using scores on the ATE Sustainability Survey. I found the model recommendations were supported by my analysis. Projects are more likely to be sustained if their staffs are well prepared; they had adequate fiscal resources; the project fit well with other institutional goals and initiatives; and they had a large number of collaborations. Two other elements were important but not to the same degree; they had an effective marketing and dissemination plan, and there was a broad based clientele implementing project programs. One element of the model was not supported by my analysis; the need for abundant information, for example, evaluation reports or advisory committee recommendations. The correlation between the indicator scores and sustainability scores was zero. This result was somewhat surprising. I suggested that perhaps having information and knowing how to act on it might be two different things. The findings of this exploratory study provide some ideas on how to enhance ATE sustainability, however, further research is needed to confirm these results.

Agency
National Science Foundation (NSF)
Institute
Division of Undergraduate Education (DUE)
Type
Standard Grant (Standard)
Application #
1132099
Program Officer
Gerhard L. Salinger
Project Start
Project End
Budget Start
2011-08-01
Budget End
2014-07-31
Support Year
Fiscal Year
2011
Total Cost
$175,332
Indirect Cost
Name
Rainbow Research, Inc.
Department
Type
DUNS #
City
Minneapolis
State
MN
Country
United States
Zip Code
55408