MOSART II conducts a longitudinal study of the impact of MSP professional development activities on teacher knowledge and relevant pedagogical content knowledge. The project expands an existing secure, online assessment system to offer MSPs involved in science education professional development the capability of pre- and post-testing participants using valid and reliable instruments based on the National Research Council's Science Content Standards. MSPs participating in MOSART II benefit from automated reports profiling aggregated pre-test performance of participants and, following post-test administration, reports of gain by content standard. MSP evaluators have access to detailed data for further analysis.
MOSART II engages 22-30 MSP projects in assessing participating teachers three times over the course of a year (e.g. prior to, soon after and at the end of the academic year following intensive MSP professional development). The resulting regression models associate teacher gains with individual attributes and with key differences in MSP professional development offerings. It allows for the determination of growth or diminishment of effect following teachers' application of what they have learned to their own classroom. This study characterizes the range in degree of impact that can be expected from professional development activities and the particular program attributes that are associated with larger gains in teacher knowledge. Controlling for teacher background aids in more accurate modeling of gains and determining if particular professional development approaches work better for different groups of teachers.
MOSART II generates evidence relevant to the following research questions: 1. What are the effects of MSP professional development experiences on participating teachers': (a) subject matter knowledge (SMK), as measured by gains in performance on misconception-oriented standards-based assessments, and (b) pedagogical content knowledge (PCK), as measured by gains in the accuracy of predicting the most common misconceptions held by their students? 2. What are the characteristics of MSP professional development experiences that are associated with gains in teachers' SMK and PCK? 3. Are there groups of teachers that are more or less affected by MSP professional experiences? 4. What is the relationship between teacher SMK and PCK, as measured by these assessments?
MOSART II expands upon nearly 1500 test items developed in its previously funded Misconceptions Oriented, Standards-Based Assessment Resource for Teachers (MOSART) work, adding items of greater difficulty to the existing test forms to improve the psychometric properties for the assessment of teacher SMK. As MSP projects involve teachers with a wide range of knowledge and teaching experience, this enhancement serves to increase the sensitivity and reliability of instruments in measuring teacher knowledge. In addition to the earth, life and physical science assessment bank, a nature of science assessment component is being added.
Ultimately, the MOSART II online assessment system provides for more uniform administration, monitoring of compliance, collection of teacher background data, and allows for characterization of key MSP program differences. The use of this online system by MSPs allows unprecedented access to their standardized test results. By keeping the identity of individuals and specific MSPs confidential, aggregated scores can be used to test various hypotheses about the effectiveness of different models of professional development.
