This collaborative research project implements a comprehensive, multidisciplinary approach to the design and validation of concept inventories (CIs). The purpose is to braoden and enhance the effective use of concept inventories in STEM education for formative and summative assessment. To accomplish this goal the project will focus on a specific concept inventory, the Thermal Transport Concept Inventory (TTCI). The data will be collected from the varied undergarduate student samples. The research would develop an explicit set of facets of understanding for diagnostic measurement and reporting, would gather multi-level empirical data of student performance on the TTCI (including student protocol and interview studies, and large scale data for psychometric modeling), would revise both the items and facets based upon these empirical data analyses, and would gather further data on the revised instrument. The data on TTCI will be analyzed in terms of the underlying conceptual knowledge and skills that it taps in critical areas of science and engineering. Additional empirical data on student reasoning and performance will be analyzed applying modern psychometric analysis techniques focused on diagnostic modeling. Research outcomes expected are: (1) an improved TTCI available through the web and accompanied by interpretive tools for use by faculty; (2) an in-depth, multi-level analysis of the assessment validity and diagnostic capacity of the TTCI; (3) a comprehensive inventory of possible formative and summative uses of CIs, including what is needed by STEM practitioners to make good use of CIs; and (4) a research and development framework for designing, developing, evaluating, improving, and/or implementing CIs in other STEM areas.
In general, the study will contribute to the better understanding of student learning and more authentic diagnostic and formative assessment, leading to better learning and teaching of important concepts in STEM. The use of the TTCI will elicit student misconceptions in the thermal and transport engineering sciences such as fluid mechanics, heat transfer, and thermodynamics. The TTCI particularly focuses on concepts deemed important but difficult for undergraduate engineering students. Thus, the proposed research is likely to lead to expanded uses of the TTCI and other CIs for formative classroom purposes as well as for a variety of summative purposes. This project are both an expansion of the psychometric modeling and methods for applying diagnostic models to CIs, and an expanded assessment validity in the area of measurement and assessment of key concepts in science and engineering.
