Supported by research on students' preconceptions, particularly in science, and their need to build on the knowledge and skills that students bring to the classroom, this proposal plans to implement a facets-of-thinking perspective for the improvement of formative assessment, learning, and instruction in high school chemistry. Its goals are: (1) to identify and develop clusters of facets (students' ideas and understandings) related to key high school chemistry concepts; (2) to develop assessment items that diagnose facets within each cluster; (3) to enhance the existing web-based Diagnoser assessment system for administering items, reporting results, and providing teacher resource materials for interpreting and using the assessment data; (4) to develop teacher professional development and resource materials to support their use of facet-based approaches in chemistry; and (5) to examine whether student learning in chemistry improves in classes that incorporate a facet-based assessment system.
The proposed work builds on previously NSF-funded projects focused on (1) designing Diagnoser (ESI-0435727) in the area of physics, and (2) assessment development to support the transition to complex science learning (REC-0129406). The work plan is organized in three strands: (1) Assessment Development, consisting of (a) development and validation of 20 facet clusters related to Reactions and Interactions, and Energy Sources and Transformations during the first three years through cognitive interviews, and (b) development and validation of 20 items per facet cluster, including their administration to chemistry classes; (2) Professional Development, through which materials will be produced for a five-day teacher workshop focused on the assessment-for-learning cycle; and (3) Technology Development, to upgrade the existing Diagnoser (in physics) in order to include chemistry tasks.
Anticipated sets of products include: (a) 8-10 validated facet clusters related to Reactions and Interactions, 8-10 validated facet clusters related to Energy Sources and Transformations, and 12-20 validated items per facet clusters; (b) 1-3 elicitation questions, 1-3 developmental lessons, and 1-3 prescriptive lessons related to each Reactions and Interactions, and Energy Sources and Transformations facet clusters, and a framework for professional development using these content resources; and (c) a web-based Diagnoser for chemistry, including student assessments and teacher resource materials.
The evaluation plan consists of three components: (1) a small-scale experimental study to examine the efficacy of the use of Diagnoser with Washington and California high school chemistry students and teachers, (2) an Advisory Board to monitor and assess the work, and (3) an external evaluator who will assess the facet and item development, as well as factors affecting Diagnoser implementation. Dissemination includes free web-based availability via Diagnoser, and workshops and presentations at national meetings.
This project addresses the practical challenge for teachers of how to develop student reasoning in high school chemistry. The design approach applies a knowledge- and learner-centered focus to instruction, explicitly considering students' conceptions and using these to guide the design of instructional activities and assessments. Professional development supports teachers in building on students' diverse linguistic, cultural, and knowledge repertories as students are beginning to learn chemistry. Online diagnostic assessments help teachers gain insights into student understandings about chemistry and enact instructional decisions to address students’ learning needs. The product of this project is the formative assessment system Diagnoser Tools for Chemistry (www.diagnoser.com). Diagnoser Tools for Chemistry includes 10 conceptual clusters that specify learning goals as well as common problematic ideas that students encounter as they learn chemistry. For each of the 10 clusters, the following tools support teachers in building on their students’ ideas and engaging students in scientific practices in multiple varied contexts: questions to elicit initial student ideas to explain real-world phenomena; developmental lessons for students to carry out investigations to explore their ideas; online diagnostic question sets with auto-generated class summaries and individual student reports for teachers; and prescriptive activities to challenge students' persistent problematic ideas and in some cases carry out additional investigations based on data in the teacher reports. Intellectual Merit. The project has made contributions to the domain of chemistry and chemistry teaching including a substantive analysis of chemistry standards, curricula, and research on misconceptions to develop practical assessment tools to help high school chemistry teachers diagnose their students' understanding. Design principles documenting decisions and approaches have been shared at conferences and in publications for science educators and researchers. Findings from a quasi-experiment with six high school chemistry teachers and approximately 500 students in each of two years indicate that student learning gains were greater when teachers used Diagnoser Tools for Chemistry. Notably, the more question sets used, the higher the gain scores on pre-post assessments. Teachers also report that Diagnoser Tools for Chemistry provide data that shape their understanding of student needs in ways that can inform their instructional decisions. Broader Impacts. Project tools and materials are disseminated to teachers on the Diagnoser Tools website (www.diagnoser.com). Beyond chemistry applications, Diagnoser Tools provides resources for physics and physical science teachers. As of October 2013, Diagnoser Tools users include 7109 teachers and 218,347 students. This project supported enhancements to Diagnoser Tools, specifically the design of collaborative web authoring tools and upgrades to the content management system to facilitate future development and scalabilty of these student and teacher resources.
