The Automated Collaboration Assessment Using Behavioral Analytics project will measure and support collaboration as students engage in STEM learning activities. Collaboration promotes clarifications of misconceptions and deeper understanding of concepts in STEM which prepares students for future employment in STEM and beyond. This project aligns with the goal of the Cyberlearning for Work at the Human-Technology Frontier program to fund exploratory research that supports learners in working productively in technology-rich STEM environments. Collaboration is an important learning skill in K-12 STEM education, yet teachers have few consistent ways to measure and support students’ development in this area. This project will result in both an improved understanding of productive collaboration and a prototype instructional tool that can help teachers identify nonverbal behaviors and assess overall collaboration and engagement quality. Using nonverbal behaviors to assess engagement will decrease dependence on discourse and content-based dialogue and increase the transferability of this work into different domains. This project is particularly timely as the ability to collaborate and engage in group work are growing requirements in professional and learning settings; at the same time the very act of collaboration is being disrupted by the Coronavirus pandemic and there is a high likelihood that much of this “new normal†(social distancing; combining in-person and remote collaboration) will be with us for some time. This project will meet the urgent need currently felt by educators and educational institutions to support the development of collaboration skills among students, even as the very act of collaboration is shifting and nontraditional forms of education are taking hold.
This project is a collaboration between the Center for Education Research and Innovation (CERI) and Center for Vision Technology (CVT) at SRI International (SRI) and will capture multiple students’ actions as they work collaboratively face-to-face, both in-person and through a virtual platform. This project will use a collaboration conceptual model, multistage predictive and explainable machine learning models, and video analytics to assess and report on collaborative behaviors and interactions. The behavior analytics system will use facial expressions, body movements, and meta-information about the collaboration task to identify interactions that show how students contribute to the collaboration, individually and collectively. This 2-year project will use reliability and model prediction testing and sequential, correlation, and thematic analyses of video recordings, surveys, interviews, and student artifacts to answer the following research questions: Can machine learning models reliably assess collaboration when compared to human assessments? How do individual behaviors during collaboration lead and relate to group level interactions and collaboration quality? and Can we validate and relate the assessed collaboration behaviors to student outcomes as represented by group-generated artifacts? The intellectual merits include contributions to the advancement of two fields: (1) machine learning— by developing and exploring new algorithms that generate explainable collaboration skill assessments and teacher/student dashboards at different grain sizes of the interactions, and (2) learning sciences—by contributing a collaboration conceptual model that shows how specific skills, interactions, and behaviors correspond to collaboration quality at group and individual levels. Broader impacts of this work include increasing the availability and types of feedback presented to instructors and learners from diverse backgrounds. This will expand the settings and number of individuals who can be evaluated and supported on collaboration by making collaborative learning easier to monitor through tools that can be used by a wide audience of educators and professionals.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
