(Taken from application abstract): While clinical information retrieval (IR) systems continue to proliferate, with the growing ubiquity of bibliographic databases and the explosion of hypermedia resources on the Internet and World Wide Web, most IR evaluation is still based on the relevance-based measures of recall and precision. Though these measures provide some insight into the successful use of IR systems, they have serious limitations. Not only is it unclear whether they truly predict successful use of IR systems, it is also unknown what constitutes a """"""""clinically significant"""""""" difference when they are being compared. We have written that these measures are especially limited when evaluating the use of IR systems by health care personnel. In their place, we have proposed a method that defines success as the ability to answer clinical questions and correlates different factors about systems and users with that success. We have performed two pilot studies, with a full-text and bibliographic database respectively, which showed the feasibility of evaluating systems based on users' abilities to perform tasks, namely answer clinical questions. In this proposal, we plan to develop this approach more fully with the ultimate goal of identifying the factors associated with successful of information retrieval (IR) systems by health care professionals.
The specific aims of this project are to: 1. Develop and implement a conceptual model of the factors which influence successful use of IR systems by health care professionals 2. Develop and refine the task-oriented approach to evaluating IR systems in the simulated clinical setting 3. Use the task-oriented approach for assessing the factors that contribute most strongly to successful system use, such as question type, databases, interfaces, and clinical levels 4. Study the use of clinical IR systems by non-physician health care providers, in particular nurse practitioners (NPs) 5. Develop a test bed of procedures and clinical questions that can be utilized in the evaluation of other IR systems
| Hersh, William R; Crabtree, M Katherine; Hickam, David H et al. (2002) Factors associated with success in searching MEDLINE and applying evidence to answer clinical questions. J Am Med Inform Assoc 9:283-93 |
| Hersh, W R; Crabtree, M K; Hickam, D H et al. (2000) Factors associated with successful answering of clinical questions using an information retrieval system. Bull Med Libr Assoc 88:323-31 |
| Hersh, W; Price, S; Donohoe, L (2000) Assessing thesaurus-based query expansion using the UMLS Metathesaurus. Proc AMIA Symp :344-8 |
