This proposal is to conduct a study of the nature, prevalence and impacts of, and reasons for the occurrence of, irreproducibility and lack of replicability in published scientific and engineering research. The awardee will convene a group of experts with knowledge of a broad range of science and engineering domains and familiarity with issues associated with reproducibility and replication. The study committee will explore issues of reproducibility and replication and produce a written report. The report will describe examples of good practices, identify gaps in knowledge, and consider the roles of researchers, professional societies, journals, universities, other research institutions, and funders in advancing reproducibility and replication. The report will include recommendations for advancing reproducibility and replication in research. The scope of the study does not include biomedical research.
When concluded, the study will illuminate the issues involved in reproducibility and replicability for the many stakeholders in the science and engineering community. The study includes opportunities for community engagement and input. The study and its recommendations will likely lead to stronger scientific practice as well as set the stage for a greater awareness among various stakeholder of how science works. Additional intellectual merit includes the potential to advance the understanding of team science and to further the understanding of how scientific inquiry is currently conducted in the United States.
The primary potential broader impacts of the project are improvements in the ability of the research community to conduct research that is reproducible and replicable. A potential ancillary benefit associated with the dissemination of the results of the study is a clearer understanding by non-scientists of the nature of science and engineering research, how it moves forward, and how scientific knowledge evolves. The project will promote dialogue between different scientific and engineering fields that involves comparing approaches to the practice of research, weaknesses that reduce reliability, and successful solutions. The findings and general recommendations in the committee's final report are expected to help promulgate best practices.
