It is well known that many students find science text challenging to comprehend. Students' reading ability is partly to blame. Reading problems become most apparent when the student is faced with a challenging text for which they have knowledge deficits. Science text, in particular, has many technical terms that are difficult to ground in everyday experience. Thus, there is a need for reading comprehension interventions to improve science comprehension. This project compares the effectiveness of two types of tutoring environments in improving high-school students' ability to understand challenging science text. Both environments contain the same pedagogical content, but present it differently: as a lesson or as a game. The first environment, developed and tested over the past 5 years, is an automated reading strategies tutor called iSTART (Interactive Strategy Training for Active Reading and Thinking) that uses animated pedagogical agents to deliver interactive instruction on self-explanation and reading strategies (comprehension monitoring, paraphrasing, generating inferences). Instruction occurs in three stages with each stage requiring increased interaction on the part of the learner. Results across a wide range of studies indicate that iSTART is highly effective in improving students' ability to understand challenging science text. While effective, iSTART can be somewhat unappealing to an average high-school student in extended practice situations. While students need extended practice to master the strategies, iSTART becomes monotonous over time. To increase students' engagement, an alternative version of iSTART practice will be developed that allows students to practice iSTART strategies in a game environment. iSTART-The Game (iTG) will present the same reading strategy practice to students, but will incorporate game-based principles to enhance engagement. This project examines whether a gaming environment for learning strategies for science text comprehension more effectively sustains students' attention and engagement during training, and thereby results in improved acquisition and mastery of these strategies. In the first year of funding, an automated reading strategy tutoring system that is framed in a gaming environment will be developed. In Year 2, pilot studies will be conducted to refine the system. In Year 3, practice using the strategies will be compared in three conditions: iTG, iSTART, and a control condition. Students will practice the strategies over a period of five additional sessions after the initial training. This experiment will provide information on the potential value of iTG over time and specifically, whether iTG engages students over repeated practice sessions. The potential interactive effects of individual differences such as prior science knowledge, reading skill, and motivational levels will also be examined.
This research will contribute to better understanding of engagement as a factor in learning gains. It is predicted by the investigators that many students who do not find the standard tutoring environment sufficiently engaging will significantly benefit from the alternative approach. Most importantly, this instructional intervention should be especially valuable for those students most at risk due to having lower ability and interest in science. This research will accomplish the goal of creating a test-bed learning environment intended to improve reader engagement and advance our understanding of the potential instructional gains from such environments. The project will contribute to our understanding of the relationship between game features and engagement. It will provide students with tools that help them more effectively meet the challenges of learning from difficult science texts. In addition, this research will contribute to our understanding of the roles that cognition and emotion play in fostering learning, the specific processes involved with learning from science texts, and the complex interplay of factors such as reading strategies, knowledge, reading skill, interest, and motivation.
The goal of this project was to design and develop a game-based intelligent tutoring system (ITS). The Interactive Strategy Training for Active Reading and Thinking (iSTART) system is an ITS designed to improve students’ ability to comprehend and learn from science texts through self-explanation and reading strategy training. Prior studies investigating iSTART found significant learning gains, but also indicated that some students were disengaged during periods of extended practice. iSTART-ME (Motivationally Enhanced) was developed during the current project to improve motivation, engagement, and persistence throughout long-term practice interactions (Jackson, Boonthum, & McNamara, 2009; Jackson, Dempsey, & McNamara, 2010; McNamara, Jackson, & Graesser, 2010). The original version of iSTART progressed students from one text to another with no intervening actions. iSTART-ME is controlled through a selection menu that provides opportunities to interact with new texts, earn points, advance through levels, purchase rewards, personalize a character, and play educational mini-games (see Figure 1). These iSTART-ME features were designed to enhance students’ motivation to persist within the iSTART environment. The features were also designed with forethought to facilitate scientific inquiry within the realm of educational games by affording investigation of the effects of separate game-based components. Several studies were conducted during this project. A subset of the studies examined separate parts of the iSTART-ME system, and two studies were conducted in which the participants engaged with the entire system over multiple sessions. Across these studies, students’ enjoyment and engagement levels increased as a function of students’ exposure to the gaming elements within iSTART-ME. In short-term studies, the game-based features tended to moderately enhance motivation but deter from learning. Within the two long-term studies, learning was equivalent between iSTART and iSTART-ME, but enjoyment and motivation ratings for the game-based system continued to increase across several weeks while boredom continually decreased. The project culminated with a study including 124 high school students comparing iSTART-ME, iSTART, and a control condition. As predicted, iSTART-ME enhanced motivation. iSTART-ME had large positive effects on students’ motivation to engage with the system. Students’ daily perception of the system, enjoyment, motivation to persist, feelings of mastery, and desire for future interactions increased for those who interacted with the game-based system, but decreased for students who interacted with the traditional iSTART tutoring system. However, pretest, posttest, and retention learning measures indicated that there were equivalent learning outcomes (and gains) comparing the game and non-game versions of iSTART. Daily performance across the 8 training sessions further indicated that the students using iSTART (the non-game version) improved steadily. By contrast, students using iSTART-ME showed an initial decline in performance followed by a steep improvement. These results indicated that engaging with the game-based system was initially distracting to learning but the effects of the distractions lessened over time. This curvilinear learning trajectory found with iSTART-ME may explain mixed results reported in the previous literature on educational games. The point in time during a learning trajectory that performance is assessed may determine whether performance trends for game-based systems appear to be negative, neutral, or potentially positive. It was also predicted that the game-based intervention would be particularly helpful for those students most at risk due to having lower ability and interest in science. The results confirmed that that the students’ with lower reading ability produced lower quality self-explanations prior to training. However, the relation between reading ability and students’ ability to self-explain and use reading strategies decreased after training. At posttest, the quality of self-explanations was instead related to training condition, with both iSTART systems outperforming the control condition regardless of initial reading ability (Jackson, Boonthum-Denecke, & McNamara, 2012). Importantly, the largest improvements in self-explanation quality were found for the low ability readers within the iSTART-ME condition (and these low ability readers are the targeted end-users of the training system; see Figure 2). These results support the main goal of the project, and provide further evidence that games can be effectively integrated within a sophisticated learning environment. In summary, the evidence from the research conducted during this project indicates that students prefer to interact with the game-based learning system. The analyses across studies included both holistic assessments at posttests, as well as iterative surveys that assessed daily changes in preferences across an extended number of sessions. The findings demonstrated that the game components integrated within iSTART-ME effectively enhanced users’ experience with the tutoring system, and that most of these benefits tended to increase across time. The goal of most tutoring systems is to produce effective and enjoyable learning experiences. However, if students do not enjoy the experience, they are likely to cease or avoid further interactions, particularly if given the choice (outside a lab-based study). These results confirm that students enjoy working with game-based systems, and that over time these systems can provide learning benefits comparable to more traditional tutoring systems.
