This two-year investigation on Linking Assessment to Science Achievement (LASA) will be conducted by researchers at University of Washington. The investigators will address cognitive validity of large-scale science test items by developing a working definition of science achievement and applying it to analyze test items. In particular, the LASA project will explore what assessment methods are best suited to measure different aspects of science achievement by attending to the supports (affordances) and constraints embedded in assessment items, comparing students' performance on items, and investigating examinees' cognitive activities when they respond to those items. Classroom observations and the analysis of students' work will also be performed in order to explore the application of the model to classroom-level assessment.
The project will include three research components: (1) a logical analysis will examine the assessment items conceptually and then use a coding theme to identify what affordances and constraints each item offers to examinees and accordingly classify the item by the type(s) of knowledge it taps; (2) a cognitive analysis will analyze concurrent and retrospective verbalizations of examinees (varying in proficiency level) while they respond to the assessment items in order to determine whether examinees' inferred cognitive activities and their strategies correspond to the item classifications; and (3) a statistical analysis will examine and compare statistical models of item scores to determine whether items classified as measuring one type of knowledge will cluster together and whether the knowledge factors will be associated with variations in instructional practices.
Intellectual Merit: The proposed framework for science achievement draws upon three branches of the literature ((1) research on science curriculum and standards addressing what students should know and be able to do in science, (2) research on cognitive psychology addressing how students learn science, (3) and research on educational measurement addressing how students' science learning should be measured) to fully address the connections between standards, learning and instruction, and assessment. Specifically, the investigators conceptualize science achievement as four interdependent types of knowledge: (1) declarative knowledge, (2) procedural knowledge, (3) schematic knowledge, and (4) strategic knowledge. The investigators assert that, in order to assess science achievement validly, tests varying in their methods for eliciting students' performance regarding different types of knowledge should be examined.
Broader Impact: The investigators believe that the proposed project will have significant impact for science assessment in specific, and educational research and practice in general. The knowledge gained from LASA will inform researchers, educators, test publishers, and policymakers about new theoretical and practical approaches to conceptualizing and assessing students' learning in the domain of science. In addition, this proposed work will contribute to studies of human cognition and learning by exploring how test takers use knowledge and strategies to solve problems in the assessment context.
