Conventional power and sample-size (P&SS) methodology for radiological imaging experiments fails to take into account imprecise pilot-study estimates of variability among radiologists, making it likely that studies will be unacceptably overpowered or underpowered. Overestimation of sample sizes results in unneeded radiologist reading time, disease-status verification, and possible risks to subjects. Underestimation of sample sizes results in inconclusive findings. Other problems with current P&SS methodology for MRMC studies are that it has only been developed for one study design, has undergone limited evaluation, and free stand-alone soft- ware does not exist for implementing it. This proposal will significantly advance DBM/OR power and sample-size methodology by contributing a thoroughly validated P&SS methodology that effectively accounts for imprecise pilot reader-variance estimates, can be used with several designs, and can be implemented using free stand-alone software. The long-term goal is to provide a thorough statistical methodology appropriate for diagnostic radiological imaging research that accounts for both patient and reader variability. The objective of this application is to improve the power and sample size (P&SS) aspect of this methodology by pursuing the following four specific aims: (1) Develop a realistic and interpretable model for generating ROC decision data for evaluating the power and sample size methodology that emulates data from clinical studies. (2) Validate a new approach to power and sample-size calculation, 'confidence-level P&SS,'that takes into account unreliable pilot-study variance estimates. (3) Extend the methodology to include other designs. (4) Develop user-friendly, free stand-alone software for implementing the methodology.
Aim 1 is necessary because current simulation models for radiological imaging data may not accurately reflect clinical studies.
Aim 2 will validate the proposed approach using the simulation model developed in Aim1.
Aim 3 will extend the P&SS methodology to include designs which include an additional factor (e.g., CAD, radiologist experience, or reading condition) for factorial as wel as nested designs (e.g., split plot de- signs).
Aim 4 will ensure essential uniform implementation and wide-spread use of the methodology. The proposed research will significantly advance P&SS methodology. The positive impact will be to accelerate the application of biomedical technologies as a result of (1) more efficient allocation of financial and human resources for research;(2) less frequent inconclusive underpowered experiments;and (3) the ability to compute sample-size estimates for several additional useful designs. The major innovation will be the development of P&SS methodology that takes into account the lack of precision in reader variance estimates by al- lowing researchers to determine sample-size estimates that correspond not only to a specified power but also to a specified confidence level that defines the probability that the specified power will actually be realized.
By allowing radiology researchers to more precisely estimate the numbers of patients and radiologists needed for imaging experiments, the proposed research will result in more efficient utilization of financial and human resources for evaluating new imaging modalities. Thus the proposed research is relevant to the mission of the National Institute of Biomedical Imaging and Bioengineering, namely to improve health by leading the development and accelerating the application of biomedical technologies.
