Personalized instruction ? instruction that targets individual students? unique learning needs and builds upon their prior knowledge ? is critical for supporting effective science learning. The primary goal in this project is to support robust learning with personalization strategies using natural language technologies. The project is a three-institution collaboration between the University of Colorado, the University Corporation for Atmospheric Research, and the University of Utah. It has two objectives: the technology objective is to create domain-independent techniques to create personalization algorithms, and the learning science objective is to measure the effect of these algorithms on learning. The project focuses on robust learning, i.e., learning the supports transfer and the promotion of meta-cognitive skills. The subject matter is earth science and biology. The proposed techonology would operate as follows. Firstly, the system uses state-of-the-art statistical natural language processing methods to automatically process learning resources (primarily texts) in order to create a domain knowledge map. This includes automatically identifying core concepts in a treatment of the subject matter. Secondly, during learning sessions, the system would analyze students' essays to dynamically construct a domain knowledge map of the students' responses (and an assessment of student understanding). Using graph matching techniques, the system evaluates the student's response, including determining what concepts were missing or misunderstood. Finally, the system uses recommendation engine methods to suggest web resources that could help the student understand the material.
This project, by automating many of the processes to identify knowledge and key concepts, has the potential to transform learning. The system is independent of the domain of learning so it can be used for any area of science. The system also does not depend upon skilled teachers ? so it can be effectively used in under-served schools.
Objectives This project’s core objectives were two-fold: (1) to develop advanced personalized learning technologies and (2) to study to what extent and under which conditions these technologies promote robust learning of science concepts in secondary science topics such as plate tectonics, weather and climate, and biological evolution. Major Outcomes An educational recommender system – CLICK2 – was created to assess learners’ essays and rich textual responses to science questions and automatically identify potential knowledge gaps and misconceptions. Interactive learning resources are strategically recommended by CLICK2 to help learners’ address these issues and develop robust understandings of core science concepts. Computer science research examined the degree to which state-of-the-art natural language processing algorithms could be used to automatically identify knowledge gaps and misconceptions and prioritize which ones to target for further instruction. Effective prioritization is both very important and very challenging. Studies of effective teaching and learning science theory highlight the importance of targeting instruction to enable students to develop deep understanding of core ideas. Such "robust" understandings are critical for supporting learners to make inferences and to generalize their knowledge to new domains. Prioritization is computationally challenging as it required fundamental advances in NLP algorithms in order to: (1) automatically extract and compare core concepts in learning resources and student writings to identify potential misconceptions and (2) sequence potential misconceptions in terms of instructional importance and recommend resources in a pedagogically useful order. Significant NLP algorithmic advances in both identifying core concepts and prioritizing misconceptions were made in this project. Our algorithm for identifying short textual similarity won the SemEval STS 2014 competition, a leading international NLP algorithm benchmarking contest. Our algorithm for prioritizing misconceptions ranked highly in a student misconception identification task at SemEval 2013. Learning sciences research informed the project’s theoretical and empirical approaches. Conceptual change theory informed the design of the CLICK2 user interface: its principals guided both how the interface highlighted potential gaps and misconceptions to the learner and how the learning resource recommendations were presented. A series of controlled learning studies were conducted to measure the influence of CLICK2 on learning outcomes. Results showed that CLICK2 had a significant positive impact on learners’ conceptual understandings and metacognitive skills. Intellectual Merit This project illustrates the powerful role learning sciences can play in fostering innovation in computer science. Our overall goal – to develop cost-effective personalized learning environments – led to fundamental advances in NLP algorithms and the design of recommendation system interfaces. The CLICK2 algorithms demonstrated very high accuracies in leading NLP competitions and have direct applications to many other problems in computer science, such as text summarization, unstructured data mining, dialog systems, and question answering. The CLICK2 interface advanced the state-of-the art in recommendation systems by demonstrating the utility of cognitively-informed theory for designing effective computer-human interaction models. Broader Impacts This project has contributed to advancing STEM learning for traditionally under-represented populations in two ways. First, CLICK2 is designed to tailor instruction to individual student needs. This capability is particularly critical for diverse students, who have widely varying background knowledge, life experiences, and linguistic skills. Scalable and effective online instructional methods that acknowledge and build on these differences are increasingly needed as our nation’s classrooms become more diverse and more opportunities for student learning move online. Second, this project contributed to the professional development of K12 STEM teachers, including those serving large numbers of diverse students, through their involvement in a series of participatory design workshops. In these workshops, teachers learned about advanced cyberlearning tools and learning sciences’ theories, and developed their capacity to serve as partners in educational technology design and research.
