In response to an increasing need for faculty members and departments to perform more thorough assessments of student learning, this project is conducting research on and developing tools to perform assessments of student learning in undergraduate chemistry courses. The overarching goal of the project is to provide chemistry instructors access to data warehouses of student performance on chemistry test items that can be queried to compare the performance of their students to the performance of the entire population in the database. This work is leveraging long-standing capacity in comparative assessment within chemistry education, as provided through norm-referenced examinations produced by the American Chemical Society Examinations Institute, with software developers, whose expertise led to the creation of the Online Web-based Learning (OWL) system used by 200,000 students of chemistry.
Intellectual Merit: Conducting psychometric research and developing semi-automated methods for instructors to align test items to a content criterion template are facilitating significant progress in both chemistry education and psychometrics. Specific project activities include producing an anchoring concepts content map (ACCM) for the entire undergraduate chemistry curriculum, creating a portal that allows chemistry instructors to connect and dynamically align their own test items to the ACCM and establishing a mechanism for instructors to compare the performances of their own students on their locally-written test items with national data sets of student performance. Developing the requisite software tools advances understanding in several key areas, such as architecture for test item meta tags, efficiency of test data warehousing, user-friendly query tools for assessment development and effectual data organization and visualization in assessment outcomes reports for users.
Broader Impacts: This work is developing practical tools for chemistry instructors from all institution types to improve their marshaling of evidence with regard to student learning in chemistry. As the database of student performance on assessments grows, data will permit for more complete and meaningful comparisons of the efficacy of curricular and pedagogical reforms. Outcomes from this project can be used to inform science education research across all disciplines.
