In this three-year project, we aim to apply state-of-the-art information analysis technologies to assist the production of systematic reviews and meta-analyses that are increasingly being used as a foundation for evidence-based medicine (EBM) and comparative effectiveness reviews. We plan to develop a human guided computerized abstract screening tool to greatly reduce the need to perform a tedious but crucial step of manually screening many thousands of abstracts generated by literature searches in order to retrieve a small fraction potentially relevant for further analysis. This tool will combine proven machine learning techniques with a new open source tool that enables management of the screening process. This new technology will enable investigators to screen abstracts in a small fraction of the time compared to the current manual process. It will reduce the time and cost of producing systematic reviews, provide clear documentation of the process and potentially perform the task more accurately. With the acceptance of EBM and increasing demands for systematic reviews, there is a great need for tools to assist in generating new systematic reviews and in updating them. This need cannot be more pressing. The recent passage of the American Recovery and Reinvestment Act and the $1.1 billion allocated for comparative effectiveness research have created an unprecedented need for systematic reviews and opportunities to improve the methodologies and efficiency of their conduct. We herein propose the development of novel, open-source software to help systematic reviewers better cope with these torrents of data. The research and development of this tool will be carried out by a highly experienced team of systematic review investigators with computer scientists at Tufts University who began to collaborate last year as a result of Tufts being awarded one of the NIH Clinical Translational Science Awards (CTSA). We will pursue dissemination of the new technology through numerous channels including, but not limited to publication, presentation at conferences, exploring interest in its adoption by the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program, Cochrane Collaboration, CTSA network, and other groups conducting systematic reviews, and production of tutorial material.
Our aims are: 1. Conduct research to design and implement a semi-automated system using machine learning and information retrieval methods to identify relevant abstracts in order to improve the accuracy and efficiency of systematic reviews. 2. Develop Abstrackr, an open-source system with a Graphical User Interface (GUI) for screening abstracts, that applies the methods developed in Aim 1 to automatically exclude irrelevant abstracts/articles. 3. Evaluate the performance of the active learning model developed in Aim 1 and the functionality of Abstrackr developed in Aim 2 through application to a collection of manually screened datasets of biomedical abstracts that will subsequently be made publicly available for use as a repository to spur research in the machine learning and information retrieval communities.
Systematic reviewing is a scientific approach to objectively summarizing the effectiveness and safety of existing treatments for diseases, a prerequisite for informed healthcare decision-making. Systematic reviewers must read many thousands of medical study abstracts, the vast majority of which are completely irrelevant to the review at hand. This is hugely laborious and time consuming. We propose to build a computerized system that automatically excludes a large number of the irrelevant abstracts, thereby accelerating the process and expediting the application of the systematic review findings to patient care.
| Wallace, Byron C; Small, Kevin; Brodley, Carla E et al. (2012) Toward modernizing the systematic review pipeline in genetics: efficient updating via data mining. Genet Med 14:663-9 |
| Wallace, Byron C; Trikalinos, Thomas A; Lau, Joseph et al. (2010) Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics 11:55 |
