The City of Chicago will embark on an innovative summer of learning (CSOL) in the summer of 2013, initiated by the Mayor?s office. CSOL engages a variety of youth serving oroganizations across the city in both self-paced and face-to-face STEM learning activites enabling students to earn STEM badges.
This RAPID project creates an approach for tracking STEM program participation among young people by using youth's badging and online participation histories. While there is considerable interest and enthusiasm around the idea of STEM badges, there is actually a relatively thin set of evaluation/research findings around badges. This project can begin to fill in a piece of that gap by providing information on youth?s participation in CSOL, the degree to which social media plays a role, and a small follow-up study around youth's continued interest and engagement in the beginning of the school year. The team will use the data to look at patterns for student engagement that can lead to profiles of participation by different types of students along with patterns of behavior. All data will be made publicly available and will enable researchers to engage in inquiry about the impact of STEM informal learning participation on longer-term STEM outcomes. The evaluation study can lead to a useful research and evaluation infrastructure that can build knowledge about the impact of summer learning on youth's interest and engagement in STEM.
The 2013 Chicago Summer of Learning (CSOL, http://chicagosummeroflearning.org) was a collaboration of youth-serving organizations across the city of Chicago focusing on summer time learning initiatives with a focus in science, technology, engineering, arts, and math (STEAM). The Digital Youth Network (DYN) was responsible for creating self-paced online activities, and for offering online learning pathways on the Remix platform, made up of smaller scaffolded challenges within a particular topic area. DYN was also responsible for offering face-to-face programming in various school and community locations to assist youth in moving through the learning pathways and for coordinating with other groups offering CSOL learning pathways within Remix. Given the short summer time frame of the initiative and the fact that DYN was involved with multiple face-to-face and online program implementations of CSOL, we received funding for an NSF Rapid grant to design and implement research to explore the use and impact of the CSOL learning opportunities by and for participants. Broad research questions include: What characteristics, participation patterns, and profiles do we see among youth and how can these patterns inform the design of more effective face-to-face activity structures and online self-paced learning experiences? How do these patterns related to student interests and attitudes toward STEAM? What kinds of analysis and representations of online data can be used by practitioners, youth, and parents to inform understanding of youth learning and progress within CSOL? How do individual interests and awareness of badge progress of friends school community and the larger community of youth influence youth patterns of participation? I. Accomplishments Meetings and organization The research team holds weekly project meetings that include logistics, updating progress, presenting recent analysis, and time for collaborative work. Established shared spaces for online collaboration (Dropbox and Evernote). We have made connections with project consultants, including Professor Brigid Barron (Stanford University School of Education, http://youthlab.stanford.edu) and research groups doing related groups in other settings (e.g. New York University Hive Research Lab, http://hiveresearchlab.org/) Development of research methodology To answer the research questions we developed new measures (quantitative (e.g. youth surveys of STEAM interest and in and out of school learning networks) and qualitative (e.g. mentor reflection interviews) and methodologies for implementation and analysis, including: Participation analysis. CSOL participation patterns online, including social network analysis, evidence of progress and persistence, and profiles of online behaviors. Pre-post analysis: STEAM learning attitudes and access to learning opportunities before and after participation in CSOL. Comparison study: Differences of STEAM learning attitudes and access to learning opportunities between youth who did and did not participate in CSOL. Comparison study: Looking at similarities and differences of participation and learning networks among the different CSOL face-to-face programs and also for those youth who participated but were not involved in any program. Data collection Remix participation data from 260 youth. Remix use data from youth (N=260), adult mentors (N=36), and other adult participants (researchers, administrators, N=13) from June-August 2013. Pre-summer survey data (N=219 students X participated in CSOL); post summer data (N= 45 students, all CSOL participants); and fall 2013 data (N=82, 22 participated in a face-to-face program, 22 participated on their own online, and 39 did not participate at all). Face-to-face program attendance data from distributed CSOL programs Informal reflection conversations with participating mentors (N=6) from CSOL face-to-face programs and online. Collection of program planning artifacts (staff meeting minutes, related online blogs) and CSOL project work from the online site (submitted artifacts). Preliminary analysis Descriptive memos of program implementation using collected artifacts and mentor interviews, focusing on program goals, adult roles, day-to-day processes, use of the online Remix system, and expectations for youth. Exploratory social network analysis visualizations, looking specifically at the entire CSOL instance and one DYN offered face-to-face program (the Digital Divas, a weekly workshop held for middle school girls at the DePaul CSOL drop-in space) in terms of youth-mentor interactions [using Gephi]. Operationalization of progress and learning that we can document from the Remix CSOL pathway participation data, and analysis of different profiles of students in the system (use and outcomes) across and comparing programs [using R language]. Preliminary pre-post survey analysis to look at change in indicators of interest and access to opportunities over the summer months and comparative analysis to compare these same indicators for youth from one school who did and did not participate in CSOL [using SPSS]. Beginning a way to describe and analyze the existing online learning pathways in terms of potential for youth to move through, including assignments, content, resources, and other supports. Preliminary visualizations Development of visualizations of youth activity at a variety of levels (e.g. individual CSOL program, full community) for different foci (e.g. participation through a learning pathway, interaction within the online community, etc.)
