We propose to organize two further Critical Assessment of Genome Interpretation (CAGI) meetings, in December 2012 and December 2013. As in 2010 and 2011, the meetings will be the culmination of a community experiment to objectively assess computational methods for predicting the phenotypic impacts of genomic variation. The CAGI experiment is timely and of wide relevance due to the burgeoning availability of individuals'genomes, and the desire to interpret these for research and clinical applications. Currently, the field lacks a consensus on the absolute and relative suitability of th panoply of different methods for prediction. These meetings will provide the first large-scale assessment of the state of the art of genome variation interpretation. The outcome of the meetings will be published to ensure wide dissemination of results. In the CAGI experiments, modeled on the Critical Assessment of Structure Prediction (CASP), participants are provided genetic variants and make predictions of resulting molecular, cellular, or organismal phenotype. Datasets are expected to include rare disease, common traits and diseases, germline and somatic cancer variation, with a focus on genomes and exomes, nsSNPs, splice-affecting SNPs, and copy number variation along with other data such as transcriptomics. Independent assessors will evaluate the predictions against experimentally characterized phenotypes. A CAGI Conference is held at the end of each experiment. The specific goals of the meeting are: (1) to assess the quality of current computational methods for interpreting genomic data, and highlight innovations &progress;(2) to guide future research efforts in computational genome interpretation and build a strong community for collaboration and interaction;and (3) to disseminate results both amongst key members of the variant-phenotype prediction community at the meeting and to a broader audience via publication of results in peer-reviewed journals. The new CAGI experiments will continue the process started with the CAGI pilot in 2010, and the first full-scale CAGI experiment in 2011. The 2011 experiment yielded a total of 117 predictions on 11 datasets, from 21 groups from 18 countries. 55 people attended the December 2011 meeting, and we are disseminating results via open access publications and conference presentations. The participating community was overwhelmingly of the opinion that this experiment is necessary and should be organized again on an ongoing basis. The organizers will strongly encourage the participation of women and underrepresented minorities, and broad participation of trainees and senior scientists at the CAGI meeting. Funding is requested for awarding 19 trainee fellowships for students and postdoctoral researchers to cover registration and approximately 2/3 of their other meeting participation costs (travel and subsistence). In addition, we seek funding to subsidize registration and approximately half of meeting participation costs of the independent assessors, some data providers and scientific council members, and the organizers of the CAGI experiments.

Public Health Relevance

Genomic variation is responsible for numerous rare diseases, propensity for many common traits and diseases, drug response, and is a key characteristic of cancer evolution. At present, our ability to characterize genetic differences far exceeds our capacity to interpret it either for basic research understanding or for clinical diagnosis. The Critical Assessment of Genome Interpretation will provide an evaluation of the current state-of-the-art and help promote progress in understanding genomic variation.

National Institute of Health (NIH)
National Human Genome Research Institute (NHGRI)
Conference (R13)
Project #
Application #
Study Section
Ethical, Legal, Social Implications Review Committee (GNOM)
Program Officer
Struewing, Jeffery P
Project Start
Project End
Budget Start
Budget End
Support Year
Fiscal Year
Total Cost
Indirect Cost
University of California Berkeley
Other Basic Sciences
Schools of Earth Sciences/Natur
United States
Zip Code
Pal, Lipika R; Kundu, Kunal; Yin, Yizhou et al. (2017) CAGI4 SickKids clinical genomes challenge: A pipeline for identifying pathogenic variants. Hum Mutat 38:1169-1181
Wang, Maggie Haitian; Chang, Billy; Sun, Rui et al. (2017) Stratified polygenic risk prediction model with application to CAGI bipolar disorder sequencing data. Hum Mutat 38:1235-1239
Pejaver, Vikas; Mooney, Sean D; Radivojac, Predrag (2017) Missense variant pathogenicity predictors generalize well across a range of function-specific prediction challenges. Hum Mutat 38:1092-1108
Daneshjou, Roxana; Wang, Yanran; Bromberg, Yana et al. (2017) Working toward precision medicine: Predicting phenotypes from exomes in the Critical Assessment of Genome Interpretation (CAGI) challenges. Hum Mutat 38:1182-1192
Hoskins, Roger A; Repo, Susanna; Barsky, Daniel et al. (2017) Reports from CAGI: The Critical Assessment of Genome Interpretation. Hum Mutat 38:1039-1041
Pal, Lipika R; Kundu, Kunal; Yin, Yizhou et al. (2017) CAGI4 Crohn's exome challenge: Marker SNP versus exome variant models for assigning risk of Crohn disease. Hum Mutat 38:1225-1234
Katsonis, Panagiotis; Lichtarge, Olivier (2017) Objective assessment of the evolutionary action equation for the fitness effect of missense mutations across CAGI-blinded contests. Hum Mutat 38:1072-1084
Kundu, Kunal; Pal, Lipika R; Yin, Yizhou et al. (2017) Determination of disease phenotypes and pathogenic variants from exome sequence data in the CAGI 4 gene panel challenge. Hum Mutat 38:1201-1216
Laksshman, Sundaram; Bhat, Rajendra Rana; Viswanath, Vivek et al. (2017) DeepBipolar: Identifying genomic mutations for bipolar disorder via deep learning. Hum Mutat 38:1217-1224
Niroula, Abhishek; Vihinen, Mauno (2017) PON-P and PON-P2 predictor performance in CAGI challenges: Lessons learned. Hum Mutat 38:1085-1091

Showing the most recent 10 out of 22 publications