(Taken from application abstract): While clinical information retrieval (IR) systems continue to proliferate, with the growing ubiquity of bibliographic databases and the explosion of hypermedia resources on the Internet and World Wide Web, most IR evaluation is still based on the relevance-based measures of recall and precision. Though these measures provide some insight into the successful use of IR systems, they have serious limitations. Not only is it unclear whether they truly predict successful use of IR systems, it is also unknown what constitutes a """"""""clinically significant"""""""" difference when they are being compared. We have written that these measures are especially limited when evaluating the use of IR systems by health care personnel. In their place, we have proposed a method that defines success as the ability to answer clinical questions and correlates different factors about systems and users with that success. We have performed two pilot studies, with a full-text and bibliographic database respectively, which showed the feasibility of evaluating systems based on users' abilities to perform tasks, namely answer clinical questions. In this proposal, we plan to develop this approach more fully with the ultimate goal of identifying the factors associated with successful of information retrieval (IR) systems by health care professionals.
The specific aims of this project are to: 1. Develop and implement a conceptual model of the factors which influence successful use of IR systems by health care professionals 2. Develop and refine the task-oriented approach to evaluating IR systems in the simulated clinical setting 3. Use the task-oriented approach for assessing the factors that contribute most strongly to successful system use, such as question type, databases, interfaces, and clinical levels 4. Study the use of clinical IR systems by non-physician health care providers, in particular nurse practitioners (NPs) 5. Develop a test bed of procedures and clinical questions that can be utilized in the evaluation of other IR systems

Agency
National Institute of Health (NIH)
Institute
National Library of Medicine (NLM)
Type
Research Project (R01)
Project #
5R01LM006311-03
Application #
2897385
Study Section
Biomedical Library and Informatics Review Committee (BLR)
Program Officer
Bean, Carol A
Project Start
1997-09-30
Project End
2000-09-29
Budget Start
1999-09-30
Budget End
2000-09-29
Support Year
3
Fiscal Year
1999
Total Cost
Indirect Cost
Name
Oregon Health and Science University
Department
Type
Schools of Medicine
DUNS #
009584210
City
Portland
State
OR
Country
United States
Zip Code
97239
Hersh, William R; Crabtree, M Katherine; Hickam, David H et al. (2002) Factors associated with success in searching MEDLINE and applying evidence to answer clinical questions. J Am Med Inform Assoc 9:283-93
Hersh, W R; Crabtree, M K; Hickam, D H et al. (2000) Factors associated with successful answering of clinical questions using an information retrieval system. Bull Med Libr Assoc 88:323-31
Hersh, W; Price, S; Donohoe, L (2000) Assessing thesaurus-based query expansion using the UMLS Metathesaurus. Proc AMIA Symp :344-8