PARTICIPATING INSTITUTIONS: Concord Consortium (Lead) Purdue University
CORE AREA(s): STEM Learning/STEM Learning Environments
PROJECT DESCRIPTION Practicing science is one of the most important goals of K-12 engineering education, which is now part of the Next Generation Science Standards. Although previous research suggests that engineering design is an effective pedagogical approach to promoting science learning, there are concerns about the "design-science gap" that fails science learning in design projects. This project is delving into large quantities of process data to systematically identify bottlenecks in design processes that pose difficulties for students to apply science.
Large learner datasets are being collected from over 3,000 students in Indiana and Massachusetts through automatic, unobtrusive logging of student design processes enabled by a unique CAD tool that supports the design of energy-efficient buildings using thermodynamics and heat transfer concepts. Large data sets - consisting of fine-grained information of student actions, experimentation results, electronic notes, and design artifacts - are used to reconstruct the entire learning trajectory of each individual student. Powerful process analytics (e.g., time series analysis and association rule mining) are being developed and applied to reveal patterns and trends across student groups and knowledge domains. Through a combination of these large data sets with pre/post-tests and demographic data, this project is answering the following research questions:
RQ1: What are the common patterns of student design behaviors and how are they associated with prior knowledge, project duration, design performance, learning outcomes, and demographic factors? RQ2: How do students deepen their understanding of science concepts involved in engineering design projects? RQ3: How often and deeply do students use scientific experimentation to make a design choice?
This five-year project is starting with six small-scale studies in years 1&2 to calibrate the process analytics by comparing with classroom observations, expert evaluations, and student interviews. The process analytics will then validate the research methodology by using the Informed Design Teaching and Learning Matrix, based on a meta-analysis of literature.
BROADER SIGNIFICANCE The scale of the project will allow for greater representation of student diversity that is not readily attainable in small-scale studies. The project is contributing to the emerging fields of educational data mining and learning analytics through researching one of the most complex STEM practices -- engineering design. Computer Aided Design data possess all four characteristics of big data defined by IBM. The big data have the potential to yield direct, measurable evidence of learning at a statistically significant scale. Automation is making this research approach highly scalable and automatic process analytics is paving the road for building adaptive and predictive software for teaching engineering design. As a by-product of this project, the redacted datasets will be freely available to any researcher who is interested in mining them.
