The Evidence Games project develops a series of interactive on-line games and investigates the effect these games have on increasing middle school science students' and teachers' knowledge and skills of scientific argumentation. There are four areas of argumentation addressed by the games: (1) understanding a claim, (2) judging the evidence about a claim based on type (fact, opinion, theory, or data) and quality (objectivity, reliability or validity), (3) analyzing the reasoning applied to the claim (authority, analogy, correlation, causation, theory, principle, or generalization), and (4) evaluating the claim (rebuttals, counterarguments, sources of error, and summary). The games increase in complexity, beginning with simple claims and evidence, and advancing to a level of analyzing current scientific claims. The games allow students to engage in discourse in an on-line, virtual environment and encourage face-to-face discourse in classes about the learning activities in the games. A component of the games allows students to enter their own claims and evidence for evaluation by other students in a challenge format. The context for the game development and refinement is middle school science classes in the Kansas City, Kansas Public Schools (KCKPS). The participants are teachers and students in those classes, who represent a diverse population. The primary STEM field is that of science, although the ultimate goal is that the games have wide application and usage in final development. The primary organizations on the project team are research groups at the University of Kansas Center for Research on Learning including ALTEC (Advanced Learning Technologies in Education), the School of Education at the University of Kansas, and the Department of Psychology and Research in Education at the University of Kansas.
The research design involves the iterative development of the games including an analysis of the authentic setting in which the game will be utilized and an iterative development process in which the team, consisting of teachers, students, and development personnel, engages in a repeated process of specifying user requirements, generating an initial conceptual analysis, developing prototypes, analyzing working prototypes, and implementing the games in classrooms. The project targets eight teachers and a broad range of middle school students in KCKPS. During the development in years one and two, four to five teachers and their students act as co-developers in each of the two phases. In the pilot phase, eight teachers with 16-24 classes participate. The researcher team collects data using a variety of established protocols ranging from pre- and post-tests to measures of scientific discussion to subjective assessment.
The products include a series of games, and findings on the usability, feasibility, fidelity, and efficacy of using a series of sub-games to support students' ability to analyze claims. This proof of concept study reaches approximately 500 students in the third year of the study in the classes of the eight participating teachers. The Evidence Games project addresses the National Science Education Standards of inquiry and the nature of science addressed in middle school, which include sophisticated, critical thinking and analysis skills, and prepares students to understand more advanced ideas in science, technology, engineering, and mathematics.
The Evidence Games project designed, developed, and evaluated the effectiveness of an online, multiplayer, competitive game, Reason Racer, in engaging middle school students in the difficult skill of scientific argumentation. Understanding and being able to correctly apply components of argumentation are important higher-order thinking and reasoning skills found across all grades. Students need to know how to think analytically when confronted with an argument, evaluate the quality of the evidence, and discuss and defend their claims with appropriate reasoning. The project was funded as a proof-of-concept grant to determine whether a game format involving rapid and competitive play, immediate feedback, high rates of responding, autonomy, and online-chat could provide an experience that would help students learn how to engage in scientific argumentation and discourse. The project collaborated with students and teachers from Argentine Middle School in Kansas City, Kansas as participant designers. The process of including this diverse group of students and teachers from the beginning use of paper-prototypes through to the finished working products ensures that the final game and resources can easily and effectively be used as a part of middle school science teaching and learning. The process also increases the likelihood that the game will engage students with different backgrounds and abilities. The team developed the Reason Racer game and supporting resources for teachers that include a dashboard with a comprehensive view of individual student and class performance across all aspects of the game and online resources to support the instructional integration of the Reason Racer game and argumentation into science classes. The final Reason Racer game contains four parts, each specifically designed to engage middle school students through interesting content, competition, autonomy, decision-making, feedback, collaboration, and discourse. When setting up play for students, the teacher assigns the game to a class by selecting from 40 different scenarios covering topics in earth and space, life, physical, technology, and engineering sciences. The different scenarios populate the content of the game’s challenges, the PitStops. The teacher makes this selection through the teacher portal (Figure 1), which also allows uploading of class rosters, viewing students’ performance data, and editing scenario assignments. The first part of the game orients players through a humorous 30-second video about the content of a particular scenario and elements of scientific argumentation (Figure 2). The second part involves players in a competitive, multiplayer, rally-type race, alternating between challenges, or PitStops, and racing segments across a variety of racecourses. The PitStops contain the content of the game, requiring students to identify or make decisions about claims, evidence, reasoning, and challenges. They require actions that are common to fast-paced games, such as matching, ranking, sorting, and discriminating, all within a rate-based game interface. Students attempt to move through each PitStop as quickly, and with as few errors, as possible. Figure 3 shows six PitStops from one scenario as an example. The competitive racing component (Figure 4) is completed between each PitStop, where students navigate various racing tracks with turns and obstacles as quickly as possible to move to the next PitStop. The third part of the game involves decision-making. Students read a brief article, about 400 words, to review the content they just encountered while in the PitStops. They move through the article at their own pace by using the space bar to scroll through the text. Their task (Figure 5) is to decide whether to accept, reject, or withhold their decision about the claim, and write a justification for their decision. This comment is inserted into the final part of the game, the discourse portion (Figure 6). After making a decision and entering a statement supporting that decision, players race to the end of the game and receive their scores. The scores also provide the players with achievements that allow them to be more competitive during the next play session. The fourth part of the game (Figure 6) engages a player in discourse with the other players about their decisions. During this discussion, students are encouraged to add points to players who make quality comments, and subtract from those who do not address the claim, evidence, or content of the article. At the end of Year Three, the project provided evidence supporting the use of Reason Racer during middle school science instruction. Students who played the game improved in every aspect of argumentation skill and judgment, and were more confident and motivated to make judgments about science claims. Also, students who discussed their judgments through the chat part of the game improved in the quality of their interactions by asking more questions and defending their conclusions with reasons. Project products included Reason Racer (http://reasonracer.com), professional development resources (http://reasonracer.wikispaces.com), 40 introductory videos (www.youtube.com/user/ReasonRacer), a patent supporting peer-mediated discourse, a data set of responses for student game-play sessions, and evaluation instruments including a Test of Scientific Argumentation.
