Replicability is a core value of science to assure that the knowledge is valid. However, the culture of scientific incentives and rewards prioritizes innovation almost exclusively at the expense of replication and verification. This project supports developing a community to shift incentives so that replicability is rewarded in aging research and social-behavioral sciences more generally. The community will create, evaluate, and disseminate four projects that could shift incentives to improve replicability. The four projects are: (1) Badges to acknowledge and reward open practices: Currently, scientists earn reputation and reward for publishing their research in journals. Replicability is enhanced if other scientists can access the materials and data that were the basis for the published research. The proposed badges will attach to articles to reward authors for making their research openly available to others. (2) Disclosure standards: The replicability of research is partly contingent on how the evidence was obtained. There are some things that every report should disclose. For example: how the sample size was determined, whether there were other measures not reported, and whether the results are dependent on other measures used in the analysis. (3) Registered reports: Researchers do not conduct replications because they are hard to publish. Registered reports will introduce a publishing format in which the study design is reviewed before the data are collected. This is good for replication research because researchers can find out before the effort of data collection if the study will be publishable. (4) Checklists for research workflow: Science is complex. It is easy to forget or fail to include important information in a report, or t check for particular things when being a reviewer or editor. Checklists can be useful guides for making sure particular practices are followed.

Public Health Relevance

Replicability is a core value of science to assure that the knowledge is valid. However, the culture of scientific incentives and rewards prioritizes innovation almost exclusively at the expense of replication and verification. This project supports developing a community to shift incentives so that replicability is rewarded in aging research and social-behavioral sciences more generally.

Agency
National Institute of Health (NIH)
Institute
National Institute on Aging (NIA)
Type
Resource-Related Research Projects (R24)
Project #
5R24AG048124-04
Application #
9316428
Study Section
Special Emphasis Panel (ZAG1)
Program Officer
King, Jonathan W
Project Start
2014-09-15
Project End
2019-04-30
Budget Start
2017-08-01
Budget End
2018-04-30
Support Year
4
Fiscal Year
2017
Total Cost
Indirect Cost
Name
Center for Open Science
Department
Type
DUNS #
079116611
City
Charlottesville
State
VA
Country
United States
Zip Code
22903
McKiernan, Erin C; Bourne, Philip E; Brown, C Titus et al. (2016) How open science helps researchers succeed. Elife 5:
Nosek, B A; Alter, G; Banks, G C et al. (2015) SCIENTIFIC STANDARDS. Promoting an open research culture. Science 348:1422-5
Miguel, E; Camerer, C; Casey, K et al. (2014) Social science. Promoting transparency in social science research. Science 343:30-1
Ioannidis, John P A; Munafò, Marcus R; Fusar-Poli, Paolo et al. (2014) Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention. Trends Cogn Sci 18:235-41