Engineering - Other (59) Assessment/Research (91) System literacy skills are at the foundation of students' abilities to comprehend how a particular system works, to identify opportunities to refine or repair that system, and more generally to adapt to novel situations. Current instructional practices, built on the assumption that basic mathematical and scientific inquiry skills must be obtained before learners can model complex problems, often delay the development of these skills. In pilot studies, undergraduate engineering students have demonstrated the potential for approaching complex engineering systems. However, they typically lack the tools necessary to comprehend these systems and to notice the relationships between components that explain how and why the system performs the way it does. This conclusion follows from performance-based assessments that require learners to generate abstract representations of systems. Performance-based assessment has long been emphasized as a method for measuring and developing both cognitive and meta-cognitive skills of learners. Because the cost of implementing performance-based assessment is high, the goal of this proposal is to construct an automated dynamic formative assessment environment that develops and measures students' ability to represent, identify, and explain solutions to complex problems. This system is called: "Graphical Representations to Assess System Performance" (GRASP). The design of GRASP is being shaped by ongoing research on evaluating students' conceptual understanding of complex systems. In addition, the project is developing methods for defining engineering problems into epistemic forms and rules. These forms and rules provide a representational structure that can be used to reliably evaluate students' work and provide meaningful feedback during practice session.
Intellectual merit: The assessment system and architecture being developed for GRASP is an interface that allows users to generate graphics, receive rapid automated feedback, and refine their solutions. This system can increase students' ability to approach initial stages of solving a problem and provide TAs and instructors the opportunity to evaluate the highly generative and cognitively rich activities associated with more ill-structured problems.
Broader impact: GRASP is designed to expand the kinds of assessments for learning that can be offered to learners of all ages and in any domain. The graphical user interface provides a new approach to assessing students' ability to explain their conceptual understanding of a system using both graphical and language literacy. GRASP integrates easily with many commercially available Learning Management Systems and thus will be attractive to a large audience of users. Once GRASP is developed, it will be adapted to K-12 learning environments through the INSPIRE Institute at Purdue University.
