The National Assessment of Educational Progress (NAEP) mathematics exam renders aggregate scores based on the assumption that each test item measures a single underlying proficiency (e.g., an area of mathematics). Prior work of the investigators suggests that there may be more information that can be mined from the NAEP exam that may be used for policymaking at the instructional level, such as policymaking for schools).
Using the NAEP item response data, the investigators propose a two-fold goal: (1) to understand school and state influences on student mathematics achievement, and (2) to improve measurement and statistical analysis techniques for the evaluation of mathematics education. A considerable portion of the research will focus on the creation of measurement and statistical techniques (goal two) that will be demonstrated in the advancement of goal one analyses.
The investigators will use Item Response Theory (IRT) and Hierarchical Linear Modeling (HLM) analyses to produce their own statistically-based classification of NAEP items that, when contrasted with the existing (official) NAEP conceptually-based classification system, should reveal information that can be attributed at the school and state levels simultaneously. This will allow the researchers to use the data and their (error) variance--both in IRT and HLM--to examine how schools differ on any number of preselected, relevant policy characteristics. To do this, however, requires that the investigators build new statistical tools and will require that this very premise be tested in the form of a hypothesis.
Thus, the results of this feasibility study will determine if a test designed for assessing national (only) progress can be used to explain the relative mathematical strengths of schools. This approach could be used to evaluate policy initiatives focused at the school level using NAEP and potentially other large-scale assessment data.
