Observer (and non-observer) performance studies are routinely performed to evaluate technologies and practices in medical imaging for possible use in the clinical environment as well as during the regulatory approval process. A large fraction of these evaluations to date, including but not limited to those performed in the field of medical imaging, required task specific ROC type response (ratings) for the analyses. Recent advancements have led to the development of analytical tools that enable the comparison of data ascertained under the FROC paradigm, which may have substantial advantages over the ROC paradigm for many of the applications of interest in this field. The FROC approach may also prove to more closely follow (emulate) the actual clinical diagnostic process, hence generalizability of and inferences based on study results could have more validity (relevance) than that attainable with other approaches, the overall objective of this project is to improve our understanding of the practical (methodological) and computational (analyses) related issues that one encounters during systems' performance studies when the experimental paradigm includes FROC type ratings and the observer, a computerized scheme (e.g. CAD), or a combination of both become an integral part of the diagnostic system. Building on a new approach to analyze scoring data ascertained during the performance of ROC type studies, we propose to develop non- parametric permutation based tests to compare FROC rating datasets in a paired design. The underlying approach compares observations between all pairs of actually negative and positive findings (e.g. cancer at a specific location) to construct summary indices of performance using a non-parametric methodology. Both unconditional and conditional (when appropriate) tests will be developed and tested. The proposed, including its advantages and limitations under different study conditions, will be investigated in a comprehensive manner using both extensive simulations as experimentally ascertained datasets. ? ? ?
Showing the most recent 10 out of 14 publications
