The studies address through investigation of students in Arizona High School summer programs how students learn, understand and manipulate scientific models of the carbon cycle and the water cycle. The investigators of this project have identified problems that on-line tutors have with creating conditions for students to regulate their own behavior. Instead of taking advantage of the available learning opportunities, some students ask for hints until the tutor gives them the correct answer.
The investigators hypothesize that lasting benefits require changing students beliefs about shallow compared with deep modeling practices and breaking their modeling habits and instilling new ones. They intend to add affective learning companions to the on-line tutors. The modules use the teachable agent metaphor called Betty's Brain. The study analysis will be conducted in high school summer schools in Arizona for students that are mostly Native Americans. The students are self-selected for attending the summer school. Data collection will include student logs, verbal protocols and affect sensor data from the summer school participants (approximately 30 students). Three studies will be conducted: 1. to develop a meta tutoring tool, 2. to develop an affective learning agent with verbal and physical gestures, and 3. a measure of whether the new tool results in increases in learning. The analysis will involve causal inference and data mining of student records of their activities. The third phase of the learning will compare students who use one agent with the teachable agent; a second group will compare two agents, the teachable agent and the meta-tutor; and a third group will compare three agents. The outcome of the study is a developed and tested tool to increase student learning and statistical analysis of records maintained by the investigators of student activities with the teachable agent. The study should help indicate whether computer tutors are more effective if they have integrated features that mimic human performance.
A student’s affective state (frustrated, engaged, bored…) can be detect with a variety of sensors, including webcams, posture-sensing chairs and pressure sensitive mice. We wondered if such technology would allow us to present affective messages (e.g., congratulations, sympathy, encouragement, etc.) at exactly the right time to increase students’ motivation . To future increase the effectiveness, the messages could be presented by an agent that the student selected as a learning companion and had had a brief get-to-know-you conversation. In order to test this hypothesis, we picked a type of learning that is known to be extremely challenging and yet increasingly important given the new common core and next generation standards. The challenge is for students to learn how to construct models of systems that change over time—dynamic systems. When given such model-construction problems, students often exhibit shallow modeling behavior: guessing, copying from old models, plugging numbers from the problem statement into random places in the model, etc. How can we teach them to use deep model practices? We developed a model construction editor that gave students simple right/wrong feedback and demonstrations as they built their model. This of course allowed them to abuse the help by asking for it when they didn’t really need. Next we built a coach (called a "meta-tutor" in the literature) that observed the students’ behavior and gave them advice about how to use the help effectively and how to do deep modeling. As expected, this nagging was partially effective. When the coach was giving advice, students exhibited more deep modeling practices, better use of help and somewhat more learning. However, also as expected, when the coach’s nagging stopped, students resumed their poor practices. This is typical of earlier attempts to teach beneficial strategies to students, including attempts that don’t involve computers. The work described so far merely set the stage for what we hoped would be the main event. We hypothesized that an affective learning companion could get students to persist in using deep modeling and effective help-seeking strategies. The agent’s job was to first build rapport with the student, then increase the students’ confidence by crediting success to the students’ hard work, and pointing out that such hard work built the "muscles" of the mind, making it even easier to succeed the next time. To make the affective learning companion effective, its affective message were selected and timed to be appropriate for the students’ emotional state, as detected using a webcam, posture chair, pressure mouse and other sensors. We develop complex rules (mostly induced from data) to determine what the agent should say under what circumstances. We tested our software with high school students participating in summer schools. This is a challenging experimental setting, as the students varied widely in age, mathematical preparation and motivation. Nonetheless, despite the large variation due to incoming attributes, we believed that the affective learning companion’s engaging persona and skillful timing of messages would create a powerful enough benefit that it could be detected amidst the noise. We ran several experiments, and each taking about 2.5 hours. We randomly assigned students to two groups: using the software (including the coach) either with or without the agent. Students first solved modeling problems for 75 minutes with the help of the coach. Students in the experimental group also interacted with their affective learning companion. After a 15 minute cookie-and-water break, students solved more modeling problems with both the coach and the agent turned off. During this second phase, we expected to see the experimental group, who had interacted with their affective learning companion, more frequently use deep modeling and better help seeking than the control group. Unfortunately, we did not detect the expected persistence. Students in both groups resumed shallow modeling and poor use of help when the nagging of the coach and the cajoling of the agent were turned off. This negative result is the main intellectual merit of this project. Perhaps the benefits of the affective learning companion would have been strong enough to detect amidst the noise if students had been able to "hang out" with their agent for longer than 75 minutes. Alternatively, existing technology might not be powerful enough to change students’ motivations at all. More experimentation is clearly needed. However, we did observe a strong, unexpected benefit. The affective learning companion significantly increased the frequency of good modeling behavior and good use of help while it was operating. This is remarkable, as the coaching had already raised the frequency well above baseline. Even though we have not been able to find the key to persistence, we now have better ways to teach deep modeling practices and more effective help seeking. This is a welcome broad impact given the importance of these practices in high-school and middle school STEM learning.
