Recent technological advances have accelerated the development of biomedical research, promising high im- pact findings in both basic research and applied or clinical settings. Exploitation of the measurement data col- lected by the ever more prevalent high throughput assays, in turn, is dependent upon innovation in computa- tional biology and bioinformatics. Crucially, we will require further methodological advances for the detection and identification of novel patterns in the data that are of biomedical relevance. Such algorithmic advances can be hard to evaluate considering a lack of ground truth for truly novel discoveries. Systematic progress, howev- er, relies on regular performance evaluations. To address this open need of the research community, we are running a series of conferences on Critical Assessment of Massive Data Analysis (CAMDA) that are orga- nized as part of the annual ISMB meetings of the International Society for Computational Biology (ISCB), the leading professional organization in the field. CAMDA has become a renowned conference, specializing in ex- amining and driving the cutting edge of complex data analyses in the life sciences. It was originally founded in 2000 (Nature 411, 885. Nature 424, 610) to provide a forum for the critical assessment of different techniques used in large-scale data analysis including, but not limited to high-dimensional gene expression profiling, other -omics, and clinical data (see www.camda.info for general background).
It aims to establish the state-of-the-art in analysis methods, identifying progress as well as remaining issues, in order to highlight promising directions for future efforts. Addressing key challenges in the field, CAMDA was one of the first conferences to adopt and optimize the approach of a community-wide contest, with competing experts of the scientific community ana- lysing the same data sets. CAMDA contests focus on technically correct measurements and signal processing on one hand, and the most demanding open ended questions of biomedical inference on the other hand. CAMDA also collaborates with high-profile groups on defining effective contests, challenging the community to deeper analyses of the latest state-of-the-art benchmarks. For instance, several benchmarks generated by the MAQC/SEQC consortia were featured prominently. These consortia were coordinated by the FDA?s NCTR. In fact, CAMDA competitions have included contest data compiled together with the FDA?s NCTR from the very beginning, and regularly since 2012. Researchers worldwide are invited to take the CAMDA challenge, which has already become a prominent fixture (cf. Nature Methods 5, 569), regularly drawing 60?100 specialists from academia and industry. The results and methods of the different contributed analyses are discussed and com- pared at the conference. Selected presentations are published in a special Open Access proceedings issue in collaboration with leading modern publishers, such as F1000 Research, offering fast-track public dissemination and Open Peer Review. Delegates jointly select a winning team and runner-ups. Strikingly, the most impres- sive approaches often come from young scientists in early stages of their careers.
Advances in biomedical research underpin the successful development and establishment of more effective and personalized medicine. To effectively exploit recent progress in genome scale or high-throughput as- says, we require methodological advances also in the computational analyses of these data. We contribute independent benchmark evaluations that are key to driving state-of-the-art methodological advancement in the research community, by running a series of conferences on Critical Assessment of Massive Data Analysis (CAMDA) that are organized as part of the annual ISMB meetings of the International Society for Computa- tional Biology (ISCB), the leading professional organization in the field.
