(as submitted to NSF) CRCNS: US - Israel Data Sharing Proposal: Computational Approaches to Assess Replicability of Neurobehavioral Phenotypes Molly Bogue, Ph.D. (PI) Yoav Benjamini, Ph.D. (PI) The Jackson Laboratory Department of Statistics and Operations Research Bar Harbor, Maine USA School of Mathematical Sciences Sagol School ofNeuroscience Tel Aviv University ISRAEL Ilan Golani, Ph.D. (Co-Inv) Department of Zoology Wise Faculty of Life Sciences Sago! School of Neuroscience Tel Aviv University ISRAEL Iliana Oozes, Ph.D. (Co-Inv) Dept. of Human Molecular Genetics and Biochemistry Saclder Faculty of Medicine Sago! School of Neuroscience Tel Aviv University ISRAEL Neri Kaflcafi, Ph.D. (Co-Inv) Department of Statistics and Operations Research School of Mathematical Sciences Tel Aviv University ISRAEL Prior NSF and/or CRCNS Support: None 35 Introduction We are submitting a revised computational neuroscience proposal. Our original proposal received positive comments from reviewers last year under the Research Proposal mechanism. For example: All reviewers were very enthusiastic about the transformative potential of solving the problem being addressed in the proposal, and were also enthusiastic about the basic framework underlying the proposed solution. However, previous reviewers thought our proposal was better aligned with the Data Sharing mechanism. Unfortunately, NSF-BSF did not support Data Sharing last year. This year it does and so we are reapplying under the Data Sharing mechanism. We have revised and improved our application to address previous reviewers' concerns. Their concerns can be grouped in four categories: 1) difficulty in acquiring new data, 2) challenge in dealing with sparse data, 3) lack of specific details about our approach, and 4) misconception that The Jackson Laboratory is a for-profit organization and that this would affect data release. JAX is a not-for-profit research institution and is a world leader in providing public bioinformatics resources for the laboratory mouse without any restrictions. Background and Significance Concernsaboutreplicabilityofneurobehavioralphenotypes The scientific and lay communities have become increasingly concerned with published discoveries that are not replicable [10-12]. Prominent institutions and journals, including NIH [13], Science [14] and Nature [15], have recently announced policy positions on the subject, yet there is still confusion and debate regarding how the problem should be addressed. While documented in all scientific fields, the problem was specifically noted in preclinical research [16] including mouse neurological and behavioral studies [7]. Note: To prevent confusion, we use the term replicability for replicating results in other studies and reproducibility for reproducing conclusions within the same study [17, 18]. ANOVA Variance Replicable Prnportlon of total variance G* L ns Yes 0% 100% Gxl ns G* L * Yes 0% 100% Gxl ns G* L * Yes 0'1/o 100% Gxl ns G ns L * No 0% 100% Gxl * g1 g2 Figure 1. Illustration of the significance of the main effects of Genotype (G), Laboratory (L) and the Genotype x Laboratory (GxL) interaction. Data for four different behavioral measures are depicted in the left panel across three laboratories (color coded) for two genotypes (g). ANOVA results for each are shown where *=significant; ns=not significant. Proportion of total variance (%) is illustrated by color-coded horizontal bars. Replicability, based on the significance of GxL, is indicated on the right for each measure. Measuring quantitative phenotypes of genetically engineered mouse strains has become a central strategy for discovering mammalian gene function, and for characterizing animal models of disease in which putative cures can be researched (for reviews see [19, 20]). The International Mouse Phenotyping Consortium (IMPC) [21, 22] coordinates an international effort to phenotype thousands of mutant strains, eventually achieving a functional annotation of most of the ~20,000 protein-coding genes in the mammalian genome. An essential aspect of this work is making high- throughput phenotyping data accessible to the scientific community in public databases [23, 24]. The utility of this undertaking, however, critically depends on the ability to replicate phenotyping results in other laboratories. This large project is but one example of the general need for replicability of mouse phenotyping, which has been repeatedly raised and discussed (e.g., [27 25 ,7] with no satisfactory solution yet adopted. Any solution that is likely to be adopted by experimentalists for 36
| Bogue, Molly A; Grubb, Stephen C; Walton, David O et al. (2018) Mouse Phenome Database: an integrative database and analysis suite for curated empirical phenotype data from laboratory mice. Nucleic Acids Res 46:D843-D850 |
| Kafkafi, Neri; Agassi, Joseph; Chesler, Elissa J et al. (2018) Reproducibility and replicability of rodent phenotyping in preclinical studies. Neurosci Biobehav Rev 87:218-232 |
