"Collaborative Research: Improving Science Learning in Inquiry-Based Programs" improves science learning by students who are not achieving their potential in high quality inquiry-based programs. While programs like FOSS, STC and Insights are effective in improving science achievement within and across school districts, many children enrolled in these programs, especially underrepresented minorities and English language learners, fail to demonstrate proficiency on standardized tests of science achievement. This project aims to achieve its goal by developing a computer program, My Science Tutor, which students will use immediately following classroom science investigations to reinforce and extend concepts embedded in the investigations. The program uses a lifelike animated character to engage students in guided learning activities and tutorial dialogs that stimulate scientific reasoning. Tutorial dialogs are based on a proven technique, Questioning the Author, which challenges students to learn to integrate new concepts with prior knowledge to construct enriched mental models that can be used to explain and predict scientific phenomena. To evaluate the intervention, learning gains are compared on standardized tests of science achievement by fourth and fifth grade students in four areas of science by students randomly assigned to three groups: the computer treatment, human tutoring or continued classroom instruction. Formative assessments analyze student and teacher experiences, and learning of concepts and scientific reasoning through analysis of entries in students' science notebooks.
My Science Tutor (MyST) is an intelligent tutoring system designed to improve children’s motivation to learn science, their ability to reason and talk about science, and their science achievement. MyST features conversational interaction with a lifelike computer character, the virtual tutor Marni, in multimedia environments. Interactions with Marni are designed to scaffold learning so that children can explain the science presented in illustrations, animations or interactive simulations. Students interact with Marni in a series of approximately sixteen 30-min tutorial sessions for a science module. In related studies, two different versions of MyST were developed: 1) MyST-Structured uses structured multimedia presentations of science followed by question-answer dialogs, and 2) MyST-Conversational usess a more conversational dialog between tutor and student. This report covers only the MyST-Structured (MyST-S) study. The primary goals of the project were to determine whether the virtual tutor could produce significant learning gains on the science topics covered and whether it could be implemented in realistic school environments. Tutorials were developed for 4 FOSS modules: 1) Magnetism & Electricity, 2) Measurement, 3) Variables and 4) Water. The tutorials were aligned to the content of Full Option Science System (FOSS) modules. Each MyST-S tutorial was organized as a sequence of mini-tutorials. Marni’s tutorial dialogs were modeled on dialogs between expert tutors and children. The dialog moves were designed to help students build on their current understandings, reason about the science, and construct explanations that communicated their new knowledge. Each MyST-S session begins with Marni asking a deep reasoning question. Narrated animations are then shown to introduce science concepts. They associate the science with materials and situations likely to be familiar to most of the students. Each narrated animation sequences the presentation in terms of the underlying set of scientific concepts, with brief pauses between each, so concepts build on each other to support a complete and accurate explanation. After the media presentations, students are presented with a question and asked to give a spoken answer. The goal was make students think about the question and express their understanding in words. Students are then given four Multiple Choice response alternatives and asked to select one. The virtual tutor provides immediate feedback. The session concludes with a short spoken dialog with Marni. Assessment of the MyST-S system occurred from November to May during the 2011 – 2012 school year. One hundred eighty three students in 13 classrooms at five schools participated in the study. Of the 183 students, 114 were in a Group treatment condition interacting with the system in groups of three, and 69 were in a 1-on-1 condition interacting with the tutor individually. Students completed assessments before and after taking the module. Learning gains from pretest to posttest were compared to learning gains of students in classrooms who did not receive supplemental tutoring. The two hypotheses for the study were: 1) Students receiving computerized tutoring in groups will achieve learning gains similar to students receiving one-on-one tutoring. 2) Both groups receiving tutoring will gain more from pretests to posttest than students receiving no tutoring. When compared with a control group that received no tutoring, moderate learning gain effect sizes were found for both treatment conditions (d = .48 Group and d = .51 1-on-1). On average, students who scored lower on the pre-test benefitted more from tutoring. Students were administered a written questionnaire on their impressions of the system. Responses were extremely positive, examples: How much did Marni help you with Science? A lot – 50%, Some - 46% Did not help – 4% How much did you enjoy working with Marni? A lot – 60% Some - 37% Did not enjoy – 3% Teachers were also given a questionnaire on their impressions of the system. Again, responses were extremely positive: Myst had a positive impact on participating students? Yes – 100% No – 0% If MyST were available, would you be interested in using it? Very Interested - 80%, Somewhat interested – 20%, Not interested – 0% We believe the study also provides evidence that spoken language understanding technology is good enough for use in this type of application. Analysis of system log files from the assessment data showed that the system understood about 86% of what students were saying. The information extracted from student speech is not used to grade the student, just to help guide the interaction. This level of understanding is easily sufficient to sustain a fluent and effective dialog. Evaluation of the MyST system produced positive user experiences and significant learning gains and indicated the feasibility of integrating MyST into real world educational environments. Students and teachers reacted very positively to the system. We believe that the study results indicate that MyST can provide a lost-cost intervention that can be implemented in school environments to help students who are struggling with science.
