The goal of this project is to develop and validate a middle school physical science assessment strand composed of four suites of simulation-based assessments for integrating into balanced (use of multiple measures), large-scale accountability science testing systems. It builds on the design templates, technical infrastructure, and evidence of the technical quality, feasibility, and instructional utility of the NSF-funded Calipers II project. The assessment strand consists of multilevel (increased thinking levels) assessment designs grounded on evidence-centered principles that target practices and key disciplinary conceptual schemes, such as matter, motion, energy, and waves identified in the National Research Council report "A Framework for K-12 Science Education: Practices, Crosscutting Knowledge, and Core Ideas". The assessment model vertically links simulations (interactive with feedback to students, coaching, and reflection); curriculum-embedded assessments for formative use; unit benchmark assessment for interim summative purposes; and a set of "signature tasks" (short-term simulations on recurring problem types). Members of the Advisory Board and an Assessment Review Panel actively participate in the development and implementation of this effort. Heller Research Associates is the external evaluator. The evaluation plan addresses both formative and summative aspects.
The project's theory of action is based on model-based learning and evidence-centered design reflective of the notion that the construct of science is multidimensional, requiring (a) understanding how the components of a science conceptual system interact to produce behaviors of the system; and (b) the use of inquiry practices to investigate the dynamic behaviors and underlying components' interactions of the system. A total of eight research and development questions guide the scope of work. The questions focus on: (a) validity (substantive and technical quality) of the individual simulation assessments; and (b) classroom implementation (feasibility, fidelity, utility). The methodology for test construction and revision follows the testing standards of major professional organizations (i.e., American Educational Research Association, American Psychological Association, and National Council of Measurement in Education) through three development phases. Phase I (Assessment Development) focuses on the alignment, quality, and prototype testing, including leverage and modification of prior work, and design of new assessment suites and signature tasks. Phase II (Pilot and Validation Studies) deals with the testing of all assessments, research instruments, and study methods. Phase III (Cross-Validation Studies) substantiates the multilevel integration assessment model, cross-validates the assessments piloted in Phase II, and establishes a reliable argument that the assessments measure the intended content and inquiry practices suitable for use in district and state-level assessment systems.
Expected outcomes are: (1) a research-informed and field-tested physical science simulations-based assessment model with high potential for extended use in middle school grades; and (2) a policy brief that provides recommendations for integrating assessments into districts and state large-scale, multi-level, balanced science assessments.
