Diagnosing Teachers' Multiplicative Reasoning (DTMR) addresses the assessment component of the DR-K12 Contextual Challenges strand. Investigating knowledge that teachers need to enable students' learning and developing assessments of that knowledge are central challenges for mathematics education. One approach emphasizes correlations between amounts of teachers' knowledge and students' achievement for accountability purposes. Another, grounded in research on mathematical thinking, often uses case studies to investigate teachers' capacities for identifying and building upon opportunities in students' problem-solving strategies. Tensions exist between these approaches because instruments convenient for assessing large numbers of teachers are insensitive to capacities for reasoning, while case study methods used to investigate teachers' reasoning are not practical with large samples. The DTMR project is building a demonstration instrument both suitable for use with large samples of teachers and informative about their capacities to reason about content in ways that support students' thinking.
In particular, the DTMR project is developing and evaluating a test form that diagnoses teachers' capacities in two closely connected cases of reasoning about multiplicative relations among quantities. The first is on on measurement that often relies on multiplicatively nested levels of units when partitioning a given quantity. The second is on covariation that often relies on multiplicative relations between distinct quantities. The project is focusing on aspects of such reasoning that are interconnected and fundamental to addition and subtraction of quantities, multiplication of quantities, quotative and partitive division of quantities, and ratios of quantities. The project will consider fractions, decimals, and ratios. A main goal of the project is to address content and construct validity of the demonstration form in sufficient depth so that larger scale work and predictive validity studies may follow.
The project is developing instruments using a new class of psychometric models called cognitive diagnosis models (CDMs). Using CDMs involves specifying components of reasoning in a particular domain and then constructing test questions (typically multiple-choice) systematically so that each choice corresponds to reasoning with a different combination of those components. Drawing on the research on students' and teachers' multiplicative reasoning, the project is developing one test form of 30 to 40 items. CDM simulation studies of estimation and equating methods are also an important component of the project.
The Diagnosing Teachersâ€™ Multiplicative Reasoning (DTMR) project, funded by the NSF DRK-12 program, demonstrated new approaches to conceptualizing and measuring teachersâ€™ mathematical knowledge aligned more closely with standards-based curricular materials than past approaches have been. This required interdisciplinary research that advanced both mathematics education and psychometrics (statistical modeling for tests). The mathematical content was multiplication and division with fractions and proportional relationships, upper elementary and middle grades content that is notoriously difficult for students and, oftentimes, teachers. The broader impact of the DTMR project is best understood in terms of alarmingly low levels of mathematics achievement among U.S. K-12 students. Instruction focused on memorizing procedures for computing and for solving formulaic word problems has left many U.S. students with little understanding of the mathematics they have studied, unable to reconstruct computational methods when forgotten, and ill-prepared to solve problems in Science, Technology, Engineering, and Mathematics (STEM) disciplines or day-to-day life. A string of national reports has sounded the alarm about our economic competitiveness. An important effort to improve the preparation of U.S. students for the STEM workforce has been the development of research-based K-12 curriculum standards and instructional materials intended to enhance studentsâ€™ conceptual understanding of mathematics and engagement with mathematics as a sense-making activity. The DTMR project was driven by two features emphasized by such standards and materials. First, there is an emphasis on logical reasoning that leads to and explains correct answers. Second, there is an emphasis on the use of problem situations and drawn models of quantities (e.g., lengths and areas) as the medium with which students can develop correct computation methods with understanding. As an example, students might begin to develop the invert-and-multiply method for dividing by fractions by drawing pictures to solve the following problem: John has 1 ¼ cups of lemonade concentrate. One pitcher of lemonade requires mixing ½ cup concentrate with water. How may pitchers of lemonade can John make? The intellectual merit of the DTMR project stemmed from the fact that past measures of teachersâ€™ knowledge had treated knowledge as something that is stored and retrieved from memory, much like data stored on a computer. Such perspectives are not well-aligned with the dynamic sense-making required to solve problems like the lemonade problem. The main goal of the DTMR project was to develop a new kind of test that diagnosed teachersâ€™ capacities to reason about problems like the lemonade problem and that could be useful as a formative assessment for teacher professional development. The first major set of findings for the DTMR project resulted from a synthesis of mathematics education research on studentsâ€™ and teachersâ€™ understanding of and reasoning with fractions and proportional relationships. The synthesis identified core components of reasoning necessary to solve problems involving multiplication and division with fractions and proportional relationships. One such component is the capacity to identify appropriate units for numbers in a situation: In the lemonade problem, 1 ¼ refers to numbers of cups, ½ refers to cups per batch (a rate), and the quotient, 5/2, refers to the number pitchers. The DTMR project developed multiple-choice and constructed response test items that provided information about teachersâ€™ facility with the identified components of reasoning. The DTMR project also found that test length required breaking the targeted content into two tests, one focused on multiplying and dividing by fractions and one focused on reasoning about proportional relationships. The second major set of findings had to do technical aspects of estimating a newly developed family of psychometric models called Diagnostic Classification Models (DCMs). These models are ideal for measuring multiple components of reasoning, like those that emerged from the synthesis of the mathematics education research literature discussed above, but they had not yet been applied to practical problems. These advances in psychometric modeling were necessary for making the DTMR test of reasoning practical. The third major set of findings stemmed from administering the DTMR Fractions survey to a national sample of 990 in-service middle grades mathematics teachers and analyzing the item response using DCMs. The DTMR Fraction survey was based on four dichotomous components of reasoning that lead to 16 possible combinations of mastery, or profiles. Analysis of the data demonstrated that the DTMR Fractions survey was indeed multi-dimensional, as hypothesized, and that teachers fell into each of the 16 possible categories. (See Figure). The distribution of profiles shows that referent unit is especially difficult for teachers (profiles 9-16). The DTMR test and the dataset on the national sample are now available to other researchers. These DTMR results demonstrate that it is possible to use a large-scale test to assess not just stored knowledge, but dynamic reasoning necessary for teaching critical upper elementary and middles content with standards-based materials. Furthermore, the project provides a model for developing similar tests of reasoning (for students or teachers) in other STEM domains.