Many of the investigations that are now carried out during imaging systems' evaluations and comparisons include interpretation (reading) studies of normal and abnormal cases that are evaluated by receiver operating characteristics (ROC) or derivatives thereof. These studies are routinely performed with such a high prevalence (i.e., true-positive rates) that observer performance may be significantly affected. Differences in performance between modes (e.g., different acquisition methods, processing techniques, or displays) may also vary significantly in the clinical environment from those observed in a high prevalence laboratory experiment. Preliminary experimental data suggest that even at a relatively high prevalence (greater than 10 percent), the effect may cause as much as a 10 percent difference in observer performance indices. The purpose of this study is to investigate the """"""""prevalence effect"""""""" on observer performance in a multi-observer, multi-abnormality study. As a part of this investigation, we will attempt not only to compare observations made under different prevalence conditions varying from approximately 33 percent (approximately 1:2) to approximately 2 percent (approximately 1:50), but also to characterize performance/prevalence curves. Since no other group has attempted to characterize the """"""""prevalence effect"""""""" in a similar quantitative manner, and because observer studies are likely to continue to play a significant role in system's evaluations, this investigation is both warranted and timely.
| Gur, David; Rockette, Howard E; Armfield, Derek R et al. (2003) Prevalence effect in a laboratory environment. Radiology 228:10-4 |
| Fuhrman, Carl R; Britton, Cynthia A; Bender, Thomas et al. (2002) Observer performance studies: detection of single versus multiple abnormalities of the chest. AJR Am J Roentgenol 179:1551-3 |
