How will we know when a bioterrorist attacks us? Since the terrorist dissemination of anthrax in October of 2001, there has been concern about the nation's vulnerability to bioterrorism, that is, to terrorism by the spread of biological agents. Increasing resources have thus been devoted to surveillance systems intended to detect such attacks. One promising detection method is to monitor visits to primary care physicians. This approach relies on the fact that many potential bioterrorism agents cause early symptoms that are non-specific. If an attack can be detected through these early symptoms, then treatment, prophylaxis, and containment can be started earlier than if definitive diagnoses are required. However, detection through surveillance of early symptoms is difficult since the 'signal' of the bioterrorism must be detected against the 'noise' of naturally occurring disease. Several statistical algorithms have been proposed for this signal detection; however, relatively little is known about the relative performance of the methods. This is an important question, as substantially improved detection may be achieved by some statistical techniques relative to others, given the same data. There are two major obstacles impeding the comparison of techniques: First, bioterrorist attacks must be simulated, since they have traits that rule out simplification and generalization. We have addressed this problem by creating a complex microsimulation to describe the effects of an anthrax attack and how it would appear in a surveillance system we operate. Second, viable metrics for comparison must be created. In this case, relatively simple methods such as ARLs or ROC curves, are insufficient, since they ignore crucial features, such as timeliness of detection, variable rankings of methods for different false positive rates, and/or the number of people affected. In this application, we propose developing tools to compare statistical methods for detection of bioterrorist attack. We will explore: 1) weighted ROC curves; 2) generalized multidimensional ROC surfaces; and 3) cost-based evaluation incorporating investigation and false positive costs as well as the value of mortality and morbidity incurred and averted by each method.
| McBrien, Kerry A; Kleinman, Ken P; Abrams, Allyson M et al. (2010) Use of outcomes to evaluate surveillance systems for bioterrorist attacks. BMC Med Inform Decis Mak 10:25 |
| Kleinman, Ken P; Abrams, Allyson M (2008) Assessing the utility of public health surveillance using specificity, sensitivity, and lives saved. Stat Med 27:4057-68 |
| Mohtashemi, Mojdeh; Kleinman, Ken; Yih, W Katherine (2007) Multi-syndrome analysis of time series using PCA: a new concept for outbreak investigation. Stat Med 26:5203-24 |
