In higher education, classroom observations are typically conducted to review and evaluate an instructor's performance. There are two main problems with this approach. First, different observers have their own ideas of what constitutes "good teaching," and often rely on these personal standards in their judgments. Thus, in principle, different observers will review the same class differently. Worse, the evaluation provides suboptimal feedback to the observed instructors because it is not typically based on evidence-based methods linked to positive student outcomes.
The focus of this project is the testing of an evaluative observation tool that is more object and informative than traditional methods. This work in progress is known as the Student Participation Observation Tool (SPOT). It is being designed as a computerized observing format and protocol that will allow an observer to categorize classroom actions during an observation, resulting in detailed time-stamped information about the progression of the class. SPOT is being designed to capture data about the use of teaching practices that promote positive student outcomes, in order to be useful to both evaluators and instructors. As a design consideration, SPOT will be easy to use and therefore will have potential for widespread adoption among faculty developers and others who routinely conduct classroom observations. This exploratory project integrates research and design components to 1) develop the observation tool based on a series of design goals, and 2) evaluate the effectiveness of the tool (how well it meets the design goals). The design objective is drawing on education research and existing observation tools to develop a format and protocol that allows collecting real-time data about student and teacher behaviors that are linked to positive student learning outcomes.
The evaluation objective is to assess the usability and usefulness of the observation format and protocol to provide valid and reliable data to instructors and evaluators about classroom practices so that they may make evidence-based decisions regarding STEM education. SPOT is being developed based on a similar well-tested tool that categorizes classroom behaviors in real time. The protocol is being developed based on a review of K-12 literature and discipline-specific higher education research regarding instructor and student classroom behaviors that result in positive student outcomes. Subsequent field-testing of SPOT in a wide-range of courses at San Jose State University will be undertaken to improve usability, and establish the validity and reliability of the tool.
