Improving statistical methods to provide better classification performance and new analytical capabilities for categorical regression would be invaluable to the medical and health care research communities. Categorical regression models (binary logistic, multinomial logistic) are used extensively to identify patterns of alcohol-related symptoms, define criteria of psychiatric disorders, and assess policies regulating alcohol. However, many such models are developed with inadequate automated support to fully analyze and exploit the intrinsically probabilistic nature of their results. This is of critical importance as researchers, clinicians, and health-care administrators are many times faced with classification decisions using categorical regression models to i) identify high risk individuals or groups, ii) make clinical assessments, or iii) establish policy and treatment guidelines Commercially available statistical software provides no automated procedures to systematically estimate and test the robustness of decision threshold(s) for classification within the context of categorical regression Moreover, the capability to estimate robust confidence intervals on decision threshold(s), compare competing classifiers, or assess the presence of classifier misspecification is completely ignored. Martingale Research will develop statistical analysis tools to provide automated support that specifically addresses the classification aspects of categorical regression modeling. This Phase I study will demonstrate using datasets representative of NIAAA databases that the proposed statistical approach will 1) estimate classification decision threshold(s), 2) provide robust confidence intervals on decision threshold(s), and 3) apply an advanced model selection test for comparing competing classifiers and analyzing classifier quality. These results will demonstrate the essential technical feasibility required for further Phase II investigation and provide the foundation for developing commercially available software.
