An award has been made to San José State University, the University of Wisconsin-Madison, and the University of Colorado at Boulder to pilot a validated assessment instrument in order to collectively assess the impacts of the Biology Research Experience for Undergraduates (Bio REU) Program. The Bio REU Program has exposed talented undergraduates to scientific research for more than 20 years. Sites are funded by NSF to support 8-12 week summer programs that culminate in a scientific symposium where students present their findings. Site-specific assessment data are collected by Site program directors and then submitted to NSF through annual progress reports. These data cannot be used to determine the effectiveness and impact of the Bio REU Program as a whole, because each program uses its own specific assessment instrument. This project will adapt and test a validated, on-line assessment tool using the Undergraduate Research Student Self-Assessment (URSSA) platform. The project will: (1) pilot test an adapted instrument on a subset of students participating in the REU site programs being conducted in summer 2010; (2) revise, retest and deploy the instrument to a larger group of REU program participants in summer 2011; (3) transfer responsibility for managing the tool and data to the Bio REU Leadership Council (LC) in 2011; and (4) make the instrument available to all Bio REU programs starting summer 2012. The PI and Co-PI of this project are members of the Bio REU LC and serve as co-chairs of the Bio REU Assessment sub-committee. These individuals will take primary responsibility for working with the URSSA team to deliver the products. At least 50 REU programs are anticipated to participate in the 2010 pilot study, and an additional 100 programs are expected to participate by 2012. The pilot group of REU sites represents a variety of REU programs, with varying scientific focus and targeted participant groups. The pilot study in summer 2010 will provide data to allow the core set of questions to be refined and be more applicable to the diversity of programs (e.g. types of institutions, sub-disciplines of biology, student populations served). Bio REU Site directors interested in participating in the pilot assessment should contact Dr. Julio G. Soto (San Jose State University) at Julio.Soto@sjsu.edu, or Dr. Janet Branchaw (University of Wisconsin Madison) at branchaw@wisconsin.edu. Information about this project can found in the Bio REU web site: www.bioreu.org/.
During the funding of our collaborative project between San José State University, the University of Wisconsin/Madison, and the University of Colorado/Boulder we developed, gathered feedback from the Biology REU community of Principal Investigators and pilot tested a modified version of the Undergraduate Research Student Self Assessment (URSSA) instrument. The instrument contained a "core" set of questions to assess the effectiveness of the Biology Research Experience for Undergraduates (Bio REU) Programs. Approximately one third (28 sites) of the over 100 funded Bio REU Programs participated in the pilot survey in summer 2010, which generated 232 student responses. In 2011, the number of sites that used the instrument increased (61 sites), which generated 538 student responses. The type of participant sites represented a diverse group of institutions funded by BIO REU including: Research Institutes, Masters granting universities, and Ph.D. granting universities. Ten biology sub-disciplines were represented including: Molecular/Cellular/Developmental Biology, Integrative Biology, Marine Biology, Physiology, Ecology/Field Biology, Molecular Biophysics, Synthetic Biology, Chemical Biology/Biochemistry, Agricultural Biology, and Bioenergy. What is URRSA? The Undergraduate Research Student Self Assessment is a survey for students in Research Experiences for Undergraduates (REU) programs. The biggest users of URSSA are organizations like BIO-REU where undergraduates participate in REU’s at multiple institutions. The survey has 132 questions in 17 sections. Students rate the first four groups of questions on how much they gained in skills, understandings, and attitudes during their research experience. Students also answer questions about their research activities, their satisfaction with the program, their motivation for joining REU’s, and other demographic information. Students respond anonymously to the survey online at Salgsite.org. How is URSSA used by the BIO-REU program? We describe student ratings of their gains in understandings and skills, how many students participated in research activities (such as attending conferences), and ratings of satisfaction with program activities and mentors. Students also answered open-ended questions about the benefits of the REU experience and their plans for the future after graduating. Comparisons are made with the four core sections of URSSA. We developed these questions from extensive interviews with REU students. These indicators assess what students gain and benefit from participation in REUs. Core areas include Thinking and Working Like a Scientist, Personal Gains, Skills and Scientific Attitudes and Behaviors. We compared averages of items from these sections with the national pool of over 1200 students. To make fair comparisons, we adjusted scores for differences in demographics and other characteristics of students and institutions. What did the core indicators on URSSA show for 2010 and 2011? The core indicators for URSSA are averages (on a scale of 1-5) of student ratings in four areas. (See last page for items in each category). When we compared averages with the larger pool of URSSA users, we found statistically significant differences favoring BIO-REU groups on all core indicators. Although students did better on several indicators in 2011, no statistically significant differences were seen between 2010 - 2011 for the BIO-REU group. Did some groups gain more than others on core indicators? We saw statistically significant differences between some of the groups taking the survey: First year students gained more on Personal Gains than Seniors. Hispanic students reported gaining more than Whites on all core indicators. Students who never participated in previous REU’s ("first time students") said they gained more on the Skills indicator than students who had attended three or more times. What types of research activities do undergraduates report? BIO-REU students reported attending conferences more often than students in other programs, and planned to present and attend conferences at greater rates than students in other programs. How satisfied are students with REU programs? Students gave activities and mentors very high ratings, with averages between 4 and 5 (on a scale of 1 -5). Lower ratings were seen for some activities such as safety training, and for training for human subjects’ research. What do students say about future plans after they have completed the REU? High numbers of students in BIO-REU said they were much more or extremely more likely to participate in activities such as work in a science lab (54%-52%), enroll in a Master’s program (41%-42%), or enroll in a Ph.D. program in science, mathematics and engineering (51%) after finishing the REU. Lower numbers of students responded that they were more likely to enroll in medical school (20%), or enroll in a program for a different professional degree (16%). What did students say about their future plans? Twenty-one percent of students in 2010 and 26% in 2011 said their experience confirmed or strengthened an existing decision to attend graduate school. Twenty percent the first year, and 14% the next year said they changed their mind during the REU and did not want to attend graduate school.
