Component 2: Research Project ? Abstract The ?Research Project? component of the Center for Critical Assessment of Genome Interpretation embodies the primary scientific activities of the Center. Organizationally, the Research Project involves developing the CAGI challenges, assessing submitted predictions, and outreach to engage and educate potential predictors.
The specific aims for this component are: 1. Recruit and develop CAGI challenges. The CAGI experiment depends upon the challenge datasets donated by clinicians and researchers. We will procure datasets with clinical relevance or which provide enhanced biomedical understanding. Once a possible dataset has been identified, there is a significant investment in formulating a challenge so that assessment will both determine which approaches are most effective, and provide insight that will further advance the field. Typical research data are highly complex, with caveats and subtleties. The art of challenge design is to pose a question that is sufficiently straightforward that it can be readily understood and addressed by predictors, while incorporating enough detail to ensure accurate representation of the underlying data and ultimate research and/or clinical relevance. 2. Assess challenge predictions. Predictions are evaluated by independent assessors. Rather than aiming to determine ?winners? and ?losers,? the intent is to understand what approaches worked, and why. To both ensure fairness and to allow insight, assessment is performed in multiple stages with numerous different assessment approaches. C-CAGI will systematize the more routine assessment methodologies for consistency between challenges and across successive experiments, and will support the development of new statistical approaches for evaluating predictions. 3. Engage and educate predictors through outreach. Many researchers who could potentially contribute to genome interpretation methodologies do not have the requisite backgrounds in all of medical genetics, genomics, biochemistry and molecular biology, computer science, and statistics?yet an understanding of all of these, at some level, is necessary to succeed in CAGI. We will aim to have outreach to clinical, biological, and computer science groups to help teach them about current methods and datasets, and build teams that can work together to participate in CAGI.
Showing the most recent 10 out of 27 publications
