Biology is now an information science, increasingly dependent on the array of biological resources, such as expert-curated databases. The knowledge in these databases ultimately derives from literature, but the transfer requires experts to read, extract and structure the information for the database, which is both labor and expertise expensive. There has been significant progress in performance of text mining and information extraction tools to support the process, much of which has been demonstrated in the context of challenge evaluations and assessment. However, all of these have measured results such as precision and recall, rather than focusing of whether the tools help the intended end users. The PI will work closely with database curators to assess the impact of the tools and develop evaluation methods to guide tool developers. The measures will be used in upcoming text mining evaluations, for the first time making the evaluation process interactive and enabling the evaluation of utility and usability.
