This project falls into the New Development track of the program guidelines. An assessment tool is being developed that can be used as both a diagnostic tool and a measurement of instructional effects, with primary focus on student conceptual understanding in the geosciences. Conceptual understanding and change are being targetted for two reasons. First, conceptual understanding implies both a familiarity with content and the ability to apply it to complex questions. Second, a number of studies have suggested that prior knowledge can be as important to understanding as pedagogy. As such, students' personal understanding of Earth systems may impact the way in which they understand and retain the formal geoscience they are exposed to. The primary goal of this study is the dissemination of a reliable and valid assessment tool to geoscience faculty around the nation, for use in general education and introductory geoscience courses, as a means of both diagnosing student preconceptions and assessing one aspect of course effectiveness. This test can thus be used as a cross-course, cross-university assessment instrument and as a means for comparing a variety of instructional styles and other variables, including disparate student outcomes related to characteristics such as age and gender. This goal will be achieved by:
1. Identification of alternative conceptions of geological processes through a comprehensive literature search and interviews with students. Geoscience covers a range of interdisciplinary studies, and we have narrowed our focus to three dimensions: Earth's crust (including topographic expression and geographic expression), Earth's interior, and Earth through time. Roughly 70 student interviews will be conducted at four different institutions with dissimilar student populations; 2. Development of a multiple-choice conceptual assessment tool using commonly held misconceptions as distractors. These misconceptions will be catalogued from the student interviews described above; and 3. Qualitative and statistical validation of the assessment tool, to ensure robustness as a comparative instrument. We will ensure both validity, the ability of a test to measure a specific characteristic, and reliability, the internal consistency of the test items and test reproducibility, of this test. Reliability and validity will be ensured through initial piloting with novice students, educators and expert geoscientists, and Item Response Theory statistics.
The availability of an assessment tool involving fundamental conceptions in geology will be invaluable for university faculty interested in assessing introductory and non-major courses in the geosciences. Additionally, the use of misconceptions as distractors allows interested faculty to use the pre-test as a diagnostic tool, to determine the kinds of alternative ideas held by their students. Teachers can then modify course structures to specifically target these preconceptions. Finally, the availability of a standard test will allow geoscience faculty to compare courses at different universities, and in so doing allow a basis by which different instructors, teaching methodologies, curriculum, and technologies can be compared. This type of evaluation is critical if we are ever truly going to answer the question "What works in the geoscience classroom?"
