More than 4,000 systematic reviews are performed each year in the fields of environmental health and evidence- based medicine, with each review requiring, on average, between six months to one year of effort to complete. In order to remain accurate, systematic reviews require regular updates after their initial publication, with most reviews out of date within five years. In the screening phase of systematic review, researchers use detailed inclusion/exclusion criteria to decide whether each article in a set of candidate citations is relevant to the research question under consideration. For each article considered, a researcher reads the title and abstract and evaluates its content with respect to the prespecified criteria. A typical review may require screening thousands or tens of thousands of articles in this manner. Under the assumption that it takes a skilled reviewer 30-90 seconds, on average, to screen a single abstract, dual-screening a set of 10,000 abstracts may require between 150 to 500 hours of labor. We have shown in previous work that automated machine learning methods for article prioritization can reduce by more than 50% the human effort required to screen articles for inclusion in a systematic review. Recently, we have further extended these methods and packaged them into a web-based, collaborative systematic review software application called SWIFT-Active Screener. Active Screener has been used successfully to reduce the effort required to screen articles for systematic reviews conducted at a variety of organizations including the National Institute of Environmental Health Science (NIEHS), the United States Environmental Protection Agency (EPA), the United States Department of Agriculture (USDA), The Endocrine Disruption Exchange (TEDX), and the Evidence Based Toxicology Collaboration (EBTC). These early adopters have provided us with an abundance of useful data and user feedback, and we have identified several areas where we can continue to improve our methods and software. Our goal for the current proposal is to conduct additional research and development to make significant improvements to SWIFT-Active Screener, including several innovations that will be necessary for commercial success. The research we propose encompasses three specific aims: (1) Investigate several improvements to statistical algorithms used for article prioritization and recall estimation. We will explore promising avenues for further improving the performance of our existing algorithms and address critical technical issues that limit the applicability of our current methods (Aim 1 ? Improved Statistical Models). (2) Explore ways in which we can improve our models and methods to handle the scenario in which an existing systematic review is updated with new data several years after its initial publication (Aim 2 ? New Methods for Systematic Review Updates). (3) Investigate several questions related to scaling the system to support hundreds to thousands of simultaneous screeners (Aim 3 - Software Engineering for Scalability, Usability and Full Text Extraction).
Systematic review is a formal process used widely in evidence-based medicine and environmental health research to identify, assess, and integrate the primary scientific literature with the goal of answering a specific, targeted question in pursuit of the current scientific consensus. By conducting research and development to build a web-based, collaborative systematic review software application that uses machine learning to prioritize documents for screening, we will make an important contribution toward ongoing efforts to automate systematic review. These efforts will serve to make systematic reviews both more efficient to produce and less expensive to maintain, a result which will greatly accelerate the process by which scientific consensus is obtained in a variety of medical and health-related disciplines having great public significance.
| Beard, John D; Erdely, Aaron; Dahm, Matthew M et al. (2018) Carbon nanotube and nanofiber exposure and sputum and blood biomarkers of early effect among U.S. workers. Environ Int 116:214-228 |
| Dahm, Matthew M; Schubauer-Berigan, Mary K; Evans, Douglas E et al. (2015) Carbon Nanotube and Nanofiber Exposure Assessments: An Analysis of 14 Site Visits. Ann Occup Hyg 59:705-23 |
