The purpose of this project is to further the research and development of tools that NIEHS can use in its research evaluations of the National Toxicology Program's (NTP's) effectiveness. NIEHS staff have developed a novel methodical approach (based upon a logic model) that yields a broad assessment of NTP' s effectiveness across multiple sectors and successfully demonstrated the utility of the approach through a case study. In applying this approach to the case study, the utility and limitations of the approach were identified, including challenges to evaluating the outcomes of a research program. ORNL will research and develop automated methods to improve the NTP's assessments. Work during FY2016-2017 will focus on development of the bibliometric tool(s) (Task 1). Once the prototype is developed and successfully tested, work will begin on the web-mining tool, anticipated for FY2017-FY2018 (Task 2). Task 1 will address development of the publication mining software for citations of NTP. ORNL will demonstrate a new bibliometrics method that will (1) facilitate retrieval of research papers that cite NTP's publication in other publications using a variety of subscription-based application programs interfaces (APIs); (2) retrieve legal, government, media, and business documents that cite NTP's work will using LexisNexis API; (3) provide the ability for NIEHS to manage documentation of compiled records, including removal of self-citations by highlighting and tagging; and (4) enable exporting of data indexed by citation tags (e.g., author, year, title, journal, volume, number, page number, uniform resource locator [URL]) and tagged categories, and importing of data into visualization tools. Successful completion of these tasks should facilitate NIEHS being able to comprehensively identify the use of NTP's research/products across multiple sectors for evaluating the effectiveness of its work.
