This proposal was submitted to the Fostering Interdisciplinary Research in Education (FIRE) activity of the Research and Evaluation in Science and Engineering Education (REESE) Program, which is designed to bring together investigators from diverse disciplines.
This project, Designing Transformative Assessments for Interdisciplinary Learning in Science (DeTAILS), aims to design and implement technology-enhanced formative assessments to help college students integrate scientific knowledge and solve complex problems across disciplines. Specifically, researchers will target fundamental biological processes in physiological contexts that are closely related to physics. The assessments will be administered to college students enrolled in introductory physics, biology, physiology, and science education courses.
The project PI, trained in both physics and physics education, will conduct the research under the supervision of the mentor (co-PI; physiologist) to expand his knowledge in physiology. This experience will help the investigator better conduct further research in science assessment to promote interdisciplinary thinking, particularly in physics, physiology, and related fields.
(DeTAILS), we designed and implemented technology-enhanced assessments to assess college students’ interdisciplinary understanding in science. The project was driven by the needs to help college students develop an array of skills in order to integrate, synthesize, and transfer knowledge within and across natural science disciplines. However, typical assessment measures in college science courses focus primarily on specific disciplinary topics and cannot capture interdisciplinary understanding. As a result, college students today not only develop a fragmented understanding of science, but are also reluctant to think beyond disciplinary constraints. We addressed this challenge by experimenting the development of an innovative assessment that could elicit and indicate students’ levels of interdisciplinary understanding. The major outcomes of the project can be summarized in the following. The project advanced both the understanding of the nature of interdisciplinary understanding, and methodology and analytical tools of assessing interdisciplinary understanding effectively. Through the project, we developed a cognitive framework that delineates the four interwoven reasoning processes (integration, translation, transfer, and transformation, i.e., the IT3 framework) and their interrelationship involved in interdisciplinary understanding and problem solving. Such a practical framework is currently lacking in the field of science education, but highly demanded. The framework can be used as an analytical framework to discern patterns of interdisciplinary interactions among people from different disciplinary backgrounds. It can also be used to guide the development of interdisciplinary assessment and instructional materials. Another significant outcome of the project is that we have developed, refined, and validated an assessment instrument on college students’ interdisciplinary understanding in science based on the IT3 framework. The instrument focused on the topic of osmosis, an interdisciplinary topic covered in physics, chemistry, and biology, which is a natural fit to address students’ interdisciplinary understanding. Using Rasch analyses, the instrument showed satisfactory psychometric quality including reliability and item fit statistics. The content validity of the instrument was ensured by the team of content experts who taught the subject for years. Student comments supported implementation validity in that the assessment items were clear and the instructions were easy to follow. The success of the instrument suggests that quantitative measures can be introduced to evaluate interdisciplinary learning. We administered the instrument to a total of 1918 college students from three classes in three iterations. Besides other important findings, we found that as the students in the samples developed conceptual understanding of osmosis over years, their growth of interdisciplinary understanding was much limited compared to that of their basic knowledge understanding. This observation was also reflected in their comments on their limited interdisciplinary learning experiences. The finding calls for changes in the current practice of college science education that leaves much of the interdisciplinary knowledge integration (at least at the introductory level) to students themselves. Also, in our sample, males performed significantly better than females overall. This result echoes that actions are needed to improve females’ science performance to achieve gender equality. We also found that the animation did contribute to students’ performance positively at a statistically significant level, when controlling for other factors such as gender, class, and year. This suggests that having students expose to dynamic visualizations, even in a very limited amount of dose, can improve student performance on interdisciplinary topics significantly. In retrospect, an effective interdisciplinary partnership among science content, education, and assessment experts was key to the success of the project. The project was a collaborative effort among multiple departments and programs including physics, biology, physiology and pharmacology, large animal medicine, small animal medicine and surgery, and science education at the University of Georgia. Each of our team members contributed significantly to the development of the assessment instrument by offering different perspectives and content knowledge. To ensure that the content was scientifically accurate and targeting students’ understanding from multiple perspectives, the team went through numerous rounds of discussions and debates. Realizing that using common terminologies was so critical in interdisciplinary communication and developing interdisciplinary assessments, we conducted a textbook analysis. The result revealed that there are indeed many inconsistent uses of terminologies regarding osmosis in college textbooks. Overall, this project results in a transformative experience for all participants. It helped all the team members gain interdisciplinary perspectives and enhance their abilities to facilitate the multidisciplinary partnership and conduct further research in interdisciplinary science assessment and education. In particular, the project PI, trained in both physics and physics education, conducted independent studies under the supervision of the mentor (co-PI) and expanded his theoretical and practical knowledge in physiology. Projects like this will greatly encourage the appreciation of disciplinary cultures, facilitate interdisciplinary communication and collaboration, and produce valid measures that meet the accountability requirement more effectively.
