This 3-year project conducts laboratory based experimental research to extend and apply a multiagent intelligent hypermedia-based learning environment to detect, track, and model college students' multiple self-regulatory processes while learning biology. The hypothesis examined by the researchers is that many college students may be hampered in learning biology by their inability to self-regulate themselves. The researchers have developed a Tutor that is expected to provide cognitive, affective, and metacognitive (CAM) support. The proposed research will test experimentally different versions of the Tutor program to examine how self-regulatory processes emerge and the effects of program variations on self-regulatory behaviors.
The investigators will conduct experiments in a laboratory with college students in Memphis Tennessee and Montreal, Canada. The experiment will detect, track, and model the CAM processes in college students' learning about a complex biology topic. During the experimental session they will collect data using a remote eye-tracker to record participants' eye gaze, fixations, saccades, and regressions. The participants' verbalizations will be recorded with a headset microphone. They will use a Pressure Mouse to capture the amount of pressure placed on the mouse throughout the activity, and the Body Pressure Measurement System (BPMS) to assess gross body movements. They will revise and develop new pretest and posttest learning measures for the circulatory system which will include approximately 15 multiple-choice questions, 10 inference questions, labeling tasks, and mental model essays. The measures will assess declarative, procedural, inferential, and mental models of the circulatory system.
The research investigators will examine several theoretical, empirical, and educational questions about self regulation intended to forge new directions of science learning. The team includes psychologists, computer scientists, psychometricians, and electrical engineers. Methodologies will be incorporated from psychology, education, computer science, and electrical engineering to detect, trace, model, and assess students' CAM self-regulatory processes during learning about a complex and challenging science topic. The proposed research activities are intended to advance the science of learning, methodologies, and quantitative analysis of complex sensing data, and education, research, and evaluation, and demonstrate the power of multi-method research tools, software, and sensing devices capable of analyzing and predicting students' self-regulated learning about complex science topics.
In this four-year project, the research team conducted laboratory-based research to extend and apply MetaTutor, a multi-agent, intelligent hypermedia-based learning environment to detect, track, and model college students' self-regulatory processes while learning biology. Three primary research questions served as a foundation for the current project: What is the nature of the temporal and dynamic unfolding of cognitive, affective, and meta-cognitive (CAM) processes during learning with MetaTutor? Are there specific 'signatures’ for each CAM process and do they differ based on experimental manipulations? Are these ‘signatures’ predictive of learning (both quantitative and qualitative), and shifts in the sophistication of the deployment of CAM processes? How do CAM processes, and relations among them, influence learning with MetaTutor? To what extent do a variety of experimental manipulations lead to differences in the deployment of CAM processes and learning? A specific hypothesis examined by the researchers is that many college students may be hampered in learning biology by their inability to self-regulate. The researchers have developed a computerized tutoring system that provided cognitive, affective, and metacognitive (CAM) support to students in real-time. The research tested different versions of the MetaTutor program to examine how self-regulatory processes emerge and their subsequent impact on student learning. During this project, our team has made several contributions to the fields of educational, cognitive, learning, and computational sciences, specifically self-regulated learning and advanced learning technologies (e.g., hypermedia and multi-agent systems). From several studies collected at multiple North American universities, we have collected and analyzed substantial amounts of data related to the affective, cognitive and metacognitive processes that learners use when interacting with a multi-agent, intelligent tutoring system The data included learning outcomes, self-report measures of emotions and agent perceptions, and multi-channel data (e.g., eye-tracking, log-files, agent-learner dialogues, facial expressions of emotions, metacognitive judgments, etc.). The multi-channel data allowed the team to map out specific cognitive, metacognitive and affective processes related to self-regulate learning. For example, we discovered that the pedagogical agents’ prompts for students to self-regulate their learning increased their use of these processes and ultimately lead to higher learning outcomes. We have also found that the way in which these prompts are enacted, both by learners and pedagogical agents, contributes to differing (e.g. positive or negative) affective states (e.g., frustration, confusion), which can, in turn, affect learning outcomes differently. Based on the extensive data collected, we are in a prime position to contribute to many theoretical models related to cognitive, metacognitive and affective processes during learning. As such, the research extends several theoretical models, contributes to emerging empirical research, and addresses critical educational questions about self-regulation and academic achievement in STEM education. The sheer volume of data collected through our experimental paradigm presented us with both opportunities and challenges. To that end, we developed a series of scripts and programs to partially automatize the temporal alignment of the various data sources, and to facilitate researchers’ integration of data across conditions and analyses of event-based data. These programs and tools include the MetaTutor Emotion Annotator, MetaTutor Log Analyzer (MTLA), and Emotion Log Analyzer (ELA). Further, we developed coding schemes to increase our accuracy and precision in isolating, measuring, and analyzing the deployment of SRL processes during learning and experimented with various methods and analytical techniques to be used to examine the temporal sequencing and unfolding of SRL processes during learning with MetaTutor. As evidence of our team’s productivity, our scholarly output during the grant period included: 2011-2012: 6 books or book chapters, 7 publications in international peer-reviewed conference proceedings, and 29 paper and poster presentations in national and international conferences. 2012-2013: 1 co-edited international handbook and 2 book chapters 3 published manuscripts, 4 papers published in international (peer-reviewed) conference proceedings, and 15 paper and poster presentations in national and international conferences. 2013-2014: 9 chapters (either published, in press, or in preparation) 5 published (or in press) journal manuscripts, 5 manuscripts under review, 3 papers published in international (peer-reviewed) conference proceedings, and 28 paper and poster presentations in national and international conferences. Our data has contributed to 2 dissertations and 3 Master’s theses for graduate students working on the project. Additionally, this project provided training opportunities for numerous graduate students in the areas of experimental design, data collection, data cleaning, and advanced data analysis. Several undergraduate students who participated on this project also developed and enhanced their research skills. We have collaborated and consulted with several international colleagues during the course of the grant including individuals from Carnegie Mellon University, Vanderbilt University, Arizona State University, University of Georgia, and University of Munich. In sum, this project made significant scientific contributions to understanding the cognitive, affective, and meta-cognitive processes that underlie student learning of complex, scientific information.
