This project aims to serve the national interest by improving the scientific reasoning skills of undergraduates in general education STEM courses. Specifically, the project focuses on helping students learn to recognize scientific arguments and use evidence-based (scientific) reasoning. Introductory courses are typically the last formal exposure to science that non-science students will have. Thus, general education STEM courses have a significant role in increasing civic science literacy. To support this goal, the project will create a writing dashboard that uses a machine learning algorithm to score how well a student?s written response supports its claims with scientific evidence. The project will also develop a web browser extension that trains students to determine whether articles on the internet provide evidence to support scientific claims. Once the dashboard and web browser extension are developed in this exploratory project, the machine learning tools can be improved and deployed nationally for use by undergraduate students and instructors. These tools have the potential for significant impact on undergraduate education, since they can assist instructors with assessing and providing feedback on writing, even in large classes. Tools that can automate the process, even partially, could enhance the use of written assignments and assessments in STEM classes, thus helping students increase their reasoning and written communication skills.
This project will implement and study the efficacy of a writing dashboard and browser extension in three large introductory science courses. The dashboard will identify pairs of phrases that represent claims and evidence to support those claims. It will also score writing based on its use of jargon and its readability. The dashboard will be designed for instructors to use as a formative assessment tool that can provide constructive feedback on student writing. It will complement the instructor?s grading process, providing a vehicle for discussing attributes associated with good scientific writing. The web browser extension will help students identify evidence-based scientific claims on the internet. Using the same machine learning technology as the writing dashboard, the browser extension will identify and highlight claims and evidence in articles available online and give an overall rating for the article?s likely scientific quality, along with a rationale for the rating. The tools will be studied in three introductory science courses taken by non-science majors: astronomy, geosciences, and evolutionary biology. Students will be required to use the dashboard for three writing assignments, and they will use the browser extension for activities that require them to review and rate online scientific articles. To study potential improvements in students? scientific reasoning capacity, the project will adapt existing survey instruments and administer the revised surveys to students before and after the intervention. Instructors will be interviewed to understand the utility of the tools in the classroom. Beyond the university setting, these tools can also be used in high schools and the browser extension can be deployed in libraries and other informal settings to help improve scientific literacy and reasoning skills within the general population. This project is supported by the NSF Improving Undergraduate STEM Education Program: Education and Human Resources Program, which supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
