This project's goal is to conduct a large study involving more than 400 students at two different institutions to study the relationship between conceptual misunderstandings, code testing and development times.
Conceptual understanding is measured by engaging students in reading and annotating documents that contain both accurate and erroneous concepts, and measuring how many students detect the errors and how many iterations it takes to detect all errors. This is then compared with code debugging times with the expectation that the text engagement process will actually reduce the amount of time spent on project tasks. The project establishes a scalable model of Reading-Annotation-Visualization (RAV) that can be deployed in any of the early programming courses. RAV model proceeds by pre-analyzing the important conceptual challenges students will encounter in the programming phase and then supporting the comprehension of these concepts prior to the programming phase through annotation, visualization and classroom discussions.
This project assesses the potential of a participatory reading, discussion and learning environment to improve the reading comprehension, writing, and programming skills of computer science undergraduates. It contributes to the understanding of the relationship between reading and discussion and programming skills in computer science. The project results are disseminated via conference proceedings. RAV model and Classroom Salon are shared with any other institution willing to implement a classroom instruction model where understanding of student misconceptions early in the learning process is critical to better learning outcomes.
The goal of this project was to investigate whether the introduction of a collaborative model of learning in the classroom can actually improve the individual learning outcomes for all students. To achieve this goal, a web-based software platform, Classroom Salon (http://classroomsalon.org) or CLS, was developed and deployed. CLS is a free public web site that is part electronic textbook and part social network. In Classroom Salon instructors group students into social groups, called salons, and introduce documents, text, and videos into these salons. Students then collaborate by highlighting, annotating, and discussing (but not editing) documents and videos and cooperatively responding to questions and extending discussion threads. Classroom Salon also contains analytical tools that help instructors determine how much students are participating and specifically where they are having trouble or find interesting information. Classroom salon is a data driven tool that is built on the principles of social cognitive theory where people learn by observing what others do and do not do. CLS is available to public for creating interactive communities centered around documents and videos. The initial methodology used in this research study was based on the Read-Annotate-Visualize (RAV) model. Under RAV model, instructor must prepare detailed or summarized documents or videos that outline the important facts and observations about the concept(s) that are covered in the course. The objective was to increase engagement with content through novel annotation and visualization tools. During the first stage of the experiment, we presented conceptual documents to see if the students were able to detect incomplete or inaccurate facts. During the second stage of the RAV model we presented number of code examples, where some critical errors were intentionally made and students were asked to detect all lines of code, where a possible error may have occurred. The annotations were then aggregated to identify critical parts of the text or code examples, where most students interpret incorrectly (or correctly). By exposing all of the aggregated annotations (which we call hotspots) individual students were able to correct some of the misconceptions before critical assignments and exams. The research employed a control and experimental group and spanned two institutions, Carnegie Mellon University and Ithaca College. At Carnegie Mellon five courses were used in the research, a systems programming course, data structures course and an introductory computer science course. Experimental class sizes averaged from 30-150 students. RAV assessment was based on an analysis of all the data collected in the course to see if community annotation visualization actually reduces the debugging errors and increase comprehension of material. The assessment was primarily based on comparing debugging times of students who were part of the RAV model versus students who were in the control group. If RAV model was successful, we expected to see a reduction in code debugging among students in the experimental group. The results of our analysis from the initial use of the RAV after the first year of the project showed that RAV had a marginal positive affect on student learning. Initial results were presented in the paper Classroom Salon: A Tool for Social Collaboration, Ananda Gunawardena and John Barr, proceedings of the 43rd ACM Technical Symposium, on Computer Science Education (SIGCSE), Feb 29-March 3, 2012, Raleigh, North Carolina. Seeking deeper results, we extended the RAV model to a new flipped class model. The flip model assumes the need to increase students engagement with content outside of the class. Class time is then used for more immersive activities such as code analysis, problem solving, and project completion. Hence we modified the software significantly to make it more amenable to the flipped model. The flipped model has been employed in five computer science courses at Carnegie Mellon and at Princeton university from 2011-2013. The dissemnitation of project findings were done through number of national and international workshops and presentations. The first workshop was given at a national conference (see Using Social Networking to Improve Student Learning Through Classroom Salon, John Barr and Ananda Gunawardena, Workshop, 43rd ACM Technical Symposium, on Computer Science Education (SIGCSE), Feb 29-March 3, 2012, Raleigh, North Carolina).Other workshops were given at Learning and Brain conference, Bryn Mawr College, Carlow University and one one-day and one day-and-a-half workshops at Carnegie Mellon University. We believe the project and its findings had a broader impact on how to improve learning gains and interest in education using social learning techniques and analytics. This is evident from the classroomsalon.org site now serving over 300 instructors, 2000+ salons/courses and nearly 15,000 academic users. In addition the development of teacher training course for HP Catalyst Academy on how to flip a class using analytics has had national and international impact. This course, together with http://classroomsalon.org website will serve as valuable resources for education community for years to come.
