Since 1993, we have been engaged in developing a practical method to evaluate the impact of Decision Support Systems (DSSs) on the diagnostic reasoning of clinicians as end-users of these systems. Our research examined the effects of four independent factors: level of subject's clinical experience, geographic site, clinical cases assigned, and specific DSS employed. Several papers addressing the questions outlined in our original proposal have been presented or published; others are under review. As this work has unfolded, we have become increasingly aware of the potential broader value of the research materials themselves and the data set generated by our research subjects. We developed a library of 36 cases, covering a broad spectrum of internal medicine. In our study, each subject evaluated nine cases, first without and then with DSS consultation. At each stage the subject was asked to list up to six diagnostic hypotheses and up to three next steps in the evaluation of the patient presented. The resulting database is one of the largest assembled to address clinical reasoning in medicine encompassing 216 clinicians, at three levels of experience across three geographic sites, who in aggregate completed 1994 cases. Our proposed work is expressed as two aims that together derive full benefit from the research already completed. Via the first aim, we propose to employ our existing data set to explore a collection of research questions at the intersection of cognitive psychology and medical informatics. These questions address issues of basic psychological import, not addressed in our original study, that will also inform the design of clinical information resources. These issues include: errors in data interpretation and problem formulation, effects of clinician confidence on judgment, the dependence of DSS consultation effects on clinician characteristics, consultation effects and case characteristics, and a set of measurement and methodology issues. Under the second aim of our proposed work, we will create and maintain our set of 36 clinical cases as a reusable, calibrated resource for other researchers.

Agency
National Institute of Health (NIH)
Institute
National Library of Medicine (NLM)
Type
Research Project (R01)
Project #
2R01LM005630-05A1
Application #
6042233
Study Section
Biomedical Library and Informatics Review Committee (BLR)
Program Officer
Florance, Valerie
Project Start
1993-09-01
Project End
2001-08-31
Budget Start
2000-05-05
Budget End
2001-05-31
Support Year
5
Fiscal Year
2000
Total Cost
$283,144
Indirect Cost
Name
University of Pittsburgh
Department
Internal Medicine/Medicine
Type
Schools of Medicine
DUNS #
053785812
City
Pittsburgh
State
PA
Country
United States
Zip Code
15213
Friedman, Charles P; Gatti, Guido G; Franz, Timothy M et al. (2005) Do physicians know when their diagnoses are correct? Implications for decision support and error reduction. J Gen Intern Med 20:334-9
Hasnain, Memoona; Onishi, Hirotaka; Elstein, Arthur S (2004) Inter-rater agreement in judging errors in diagnostic reasoning. Med Educ 38:609-16
Friedman, Charles P; Gatti, Guido G; Murphy, Gwendolyn C et al. (2002) Exploring the boundaries of plausibility: empirical study of a key problem in the design of computer-based clinical simulations. Proc AMIA Symp :275-9
Elstein, Arthur S; Schwarz, Alan (2002) Clinical problem solving and diagnostic decision making: selective review of the cognitive literature. BMJ 324:729-32
Friedman, C; Gatti, G; Elstein, A et al. (2001) Are clinicians correct when they believe they are correct? Implications for medical decision support. Medinfo 10:454-8
Friedman, C P; Elstein, A S; Wolf, F M et al. (1999) Enhancement of clinicians' diagnostic reasoning by computer-based consultation: a multisite study of 2 systems. JAMA 282:1851-6
Friedman, C; Elstein, A; Wolf, F et al. (1998) Measuring the quality of diagnostic hypothesis sets for studies of decision support. Medinfo 9 Pt 2:864-8
Wolf, F M; Friedman, C P; Elstein, A S et al. (1997) Changes in diagnostic decision-making after a computerized decision support consultation based on perceptions of need and helpfulness: a preliminary report. Proc AMIA Annu Fall Symp :263-7
Elstein, A S; Friedman, C P; Wolf, F M et al. (1996) Effects of a decision support system on the diagnostic accuracy of users: a preliminary report. J Am Med Inform Assoc 3:422-8